Evidence assessment
The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits
- Source
- url:https://arxiv.org/abs/2402.17764
- Pages
- 8
Assessment outcome
Insufficient evidence to score confidently
Assessment confidence: LimitedPaper type: Uncertain or General
Why this score
Insufficient evidence to score confidently
Missing information
- Required stage claims did not produce usable output.
- Required stage evidence did not produce usable output.
- Required stage methodology did not produce usable output.
- Source record kind struct provided 0 usable records; required minimum: 1.
- Source record kind claim provided 0 usable records; required minimum: 1.
- Source record kind evidence provided 0 usable records; required minimum: 1.
- Source record kind method provided 0 usable records; required minimum: 1.
- Major claim count was 0; required minimum: 1.
- No scored criterion was produced; at least one scored criterion was required.
- No scored dimension was produced; at least one scored dimension was required.
- Assessable configured dimension weight 0 was below 70% of applicable configured weight 1.00; 0.70 was required.
Dimensions and criteria
- Not assessable
Claim-evidence alignment
general.claim_evidence_alignment
Not assessable
- Configured weight
- 0.40
- Effective weight
- Not scored
Criteria
general_claim_evidence_alignment_support_coverage
- Status
- Not assessable
- Rating
- Not assessable
- Points
- Not scored
- Configured weight
- 0.50
- Effective weight
- Not scored
general_claim_evidence_alignment_scope_and_nature
- Status
- Not assessable
- Rating
- Not assessable
- Points
- Not scored
- Configured weight
Claim alignments
No claim alignments are available.
Within-paper comparisons
No within-paper comparisons are available.
Methodology
Methodology checklist
No methodology records were extracted.
Limitations
External competitiveness was not assessed.
Missing information
- Required stage claims did not produce usable output.
- Required stage evidence did not produce usable output.
- Required stage methodology did not produce usable output.
- Source record kind struct provided 0 usable records; required minimum: 1.
- Source record kind claim provided 0 usable records; required minimum: 1.
- Source record kind evidence provided 0 usable records; required minimum: 1.
- Source record kind method provided 0 usable records; required minimum: 1.
- Major claim count was 0; required minimum: 1.
- No scored criterion was produced; at least one scored criterion was required.
- No scored dimension was produced; at least one scored dimension was required.
- Assessable configured dimension weight 0 was below 70% of applicable configured weight 1.00; 0.70 was required.
Paper context
- problem_statement
- In recent years, the field of AI has seen a rapid growth in the size and capabilities of Large Language Models (LLMs). These models have demonstrated remarkable performance in a wide range of natural language processing tasks, but their increasing size has posed challenges for deployment and raised concerns about their environmental and economic impact due to high energy consumption.
- section · page 2 · 1 The Era of 1-bit LLMs
- proposed_approach
- BitNet b1.58 is based on the BitNet architecture, which is a Transformer that replaces nn.Linear with BitLinear. It is trained from scratch, with 1.58-bit weights and 8-bit activations.
- section · page 2 · 2 BitNet b1.58
- main_contribution
- In this work, we introduce a 1-bit LLM variant, namely BitNet b1.58, in which every single parameter (or weight) of the LLM is ternary {-1, 0, 1}. It matches the full-precision (i.e., FP16 or BF16) Transformer LLM with the same model size and training tokens in terms of both perplexity and end-task performance, while being significantly more cost-effective in terms of latency, memory, throughput, and energy consumption.
- section · page 1 · Abstract
Evidence ledger
Evidence ledger
No source records are available.