1. Introduction

Weight-only quantization of Large Language Models (LLMs) aims to make LLM serving efficient by reducing parameter precision. While 4-bit methods (GPTQ [1], AWQ [2]) have seen widespread industrial adoption, advancing to binary (1-bit) or sub-binary compression presents distinct challenges. Existing binary post-training quantization (PTQ) methods employ in-place binarization with full-precision scales, resulting in effective storage costs of 2-3 bits per weight despite theoretical 1-bit representation. Conversely, binary quantization-aware training (QAT) methods achieve 1-bit compression but require end-to-end training on billions of tokens using multiple GPUs, making it impractical in resource-limited settings.

In this post, we introduce NanoQuant, a post-training quantization method achieving 1-bit and sub-1-bit weight compression. With NanoQuant, compression of a 70B model completes in 13 hours on a single NVIDIA H100 GPU, achieving 24× storage reduction while enabling inference on consumer 8GB GPUs.

2. Problem Formulation and Background

Low-Rank Binary Factorization

Standard binary quantization approximates a weight matrix $W∈\Bbb R^{(d_{out}×d_{in})}$ as:

$W≈α⋅B_{±1}$

where $B_{±1}∈\{−1,+1\}^{(d_{out}×d_{in})}$ and $α∈\Bbb R$ is a scaling factor. This parameterization is structurally bounded at 1 bit per weight. In practice, metadata encoding (scaling factors, per-group parameters, sparsity masks) inflates effective storage to 2-3 bits per weight, reducing the advantage over 2-bit integer quantization.

NanoQuant addresses this limitation through low-rank binary factorization:

$W≈s_1⊙(U_{±1} V_{±1}^T )⊙s_2^T$

where $U_{±1}∈\{-1,+1\}^{(d_{out}×r)}$, $V_{±1}∈\{-1,+1\}^{(d_{in}×r)}$, $s_1∈R^{\{d_{out}\}}$, and $s_2∈R^{\{d_{in}\}}$. This representation requires $r(d_{out}+d_{in})$ binary values plus $(d_{out}+d_{in})$ floating-point values. For appropriately chosen rank $r$, the total storage reduces to 1 bit per weight or less, with metadata overhead negligible relative to the binary matrix components.

Challenges in Binary Factorization

Optimizing the binary factorization $W≈s_1⊙(U_{±1} V_{±1}^T )⊙s_2^T$ is combinatorially difficult. The binary constraints on $U_{±1}$ and $V_{±1}$ render this a non-convex, discrete optimization problem with exponential solution space. Quantization-aware training addresses this through prolonged data exposure and end-to-end optimization. Post-training quantization operates under constrained conditions: a small calibration dataset (128 sequences in NanoQuant) and a limited optimization budget.

Weak initialization leaves substantial reconstruction error that cannot be recovered through lightweight calibration. Conversely, initialization sufficiently close to a local optimum enables modest optimization to achieve competitive accuracy. This motivates NanoQuant''s emphasis on initialization quality as a central algorithmic component.

3. Methodology

NanoQuant comprises three phases: (1) global Hessian-aware calibration; (2) sequential block-wise reconstruction with precise binary initialization; and (3) scale-only model reconstruction for global alignment.

Phase 1: Calibration and Preconditioning

In the calibration phase, activation statistics $\{z_{in}^l,z_{out}^l\}$ are collected for each linear layer l by executing a forward-backward pass on the calibration dataset. These statistics are used to construct robust diagonal preconditioners $D_{in}$ and $D_{out}$ via Kronecker-factored approximate curvature (KFAC), incorporating shrinkage regularization for numerical stability. The preconditioned target weight matrix becomes:

$ W_{target}^l←D_{out}^l \tilde W$$^l$$D_{in}^l $

These preconditioners are shared across all blocks and remain fixed during subsequent phases.

Phase 2: Sequential Block Reconstruction

Transformer blocks are compressed sequentially. Each block receives activations generated by previously quantized blocks rather than the full-precision model. This sequential processing reduces train-test mismatch by exposing each block to accumulated quantization error during initialization and refinement.

We delineate the block reconstruction process of compressing a single linear layer in a typical decoder block above. Further details for each step are mentioned below.

Step 1: Error Propagation Mitigation

The current full-precision block is fine-tuned to minimize reconstruction error given the altered input distribution from preceding quantized blocks. This step reduces error accumulation across layers by compensating for distributional shifts introduced by quantization.

Step 2: Low-Rank Binary Initialization via LB-ADMM

The core innovation in NanoQuant is Latent Binary ADMM (LB-ADMM), which initializes the low-rank binary factorization by decoupling continuous reconstruction from discrete binary constraints.

NanoQuant utilizes alternating direction method of multipliers (ADMM) to initialize the latent binary matrices and full-precision scales. Notably, we utilize a highly optimized implementation of ADMM that utilizes stabilized Cholesky decomposition for faster convergence.

Magnitude Balancing and Scale Extraction

After LB-ADMM convergence, the pre-binary variables exhibit scale ambiguity ($U$ and $V$ can be scaled inversely without changing their product). To condition subsequent optimization, NanoQuant balances magnitudes:

$ η=\sqrt{ \dfrac{‖\widehat V‖_F}{‖\widehat U‖_F} }$

where $ \widehat U$ and $\widehat V$ are the unscaled continuous proxies. Balanced factors are defined as

$ U≔η\widehat U,V≔\dfrac{\widehat V}{η} $

Scaling vectors are extracted via mean absolute value:

$ [s_1]_i = \overline{[η\widehat U]_i} , [s_2 ]_i=\overline{[\widehat V /η]_i } $

This separation ensures the scaling vectors capture magnitude information while leaving well-conditioned latent matrices for fine-tuning.

Step 3: Factorized Component Refinement

Starting from the LB-ADMM initialization, the latent factors and scaling vectors are jointly refined to minimize block-level reconstruction error. The optimization objective is:

$ \displaystyle \min_{\rm {(U,V,s_1,s_2 )}}⁡ ‖B(X_{in} - \tilde B (X_{in};sign(U),sign(V),s_1,s_2 )‖_F^2 $

where B denotes the full-precision block output and B ̃ is the quantized version. Straight-Through Estimators (STE) enable gradient flow through sign functions, allowing updates to continuous factors while simulating final binary quantization.

Because LB-ADMM provides initialization near a local optimum, refinement focuses on fine-tuning factors near the binary decision boundary rather than reconstructing the core binary structure from scratch. Empirically, sign-flip rates remain below 7% across all layers, confirming the stability of the LB-ADMM initialization.

Phase 3: Scale-Only Model Reconstruction

After block-wise quantization, all binary factors are frozen and packed into efficient integer formats. A final global calibration aligns the quantized model''s output distribution with the full-precision model by optimizing only the floating-point scaling vectors $ \{s_1^l,s_2^l\} $ for all layers l.

The optimization objective minimizes KL divergence between output logit distributions:

$ \displaystyle \min_{\rm S_{global}}⁡ ‖Logits(M(X))-Logits(\widehat M (X;S_{global} ))‖_{KL} $

where $ S_{global}∈\{s_1^l,s_2^l\}$ is the set of scaling parameters. Freezing binary weights substantially reduces memory requirements, enabling 70B model calibration on a single GPU.

4. Results

Accuracy and Storage Efficiency

NanoQuant was evaluated on five model families - Llama-2, Llama-3, Gemma 3, Qwen3, and Rnj-1 - with sizes ranging from 0.6B to 70B parameters. Calibration used 128 WikiText-2 sequences (approximately 0.26M tokens).

When using NVIDIA H100 GPUs to compress Llama-2-7B, NanoQuant outperforms most binary PTQ and QAT baselines on Wikitext2 perplexity, on a small data and compute budget.

NanoQuant also shows relatively strong performance across models from the Qwen3 family. 1-bit and sub-1-bit NanoQuant outperforms 2-bit GPTQ and 2.88-bit BiLLM [3], and 2.0-bit NanoQuant shows superior performance compared to higher-precision binary PTQ methods and 3-bit GPTQ.

Comparison with Vector Quantization

Higher-precision NanoQuant also shows competitive performance with state-of-the-art vector quantization methods, QTIP [4], AQLM [5], and PV-Tuning [6]. Notably, we only require a single GPU and a few hours, compared to other methods requiring multiple GPUs for multiple days.

Comparison with Quantization-Aware Training

NanoQuant was compared against two state-of-the-art low-rank binary QAT methods - LittleBit [7] and Double Binary Factorization (DBF) [8] - at approximately 1 bit per weight. Results demonstrate comparable predictive performance with substantially reduced computational requirements:

NanoQuant achieves superior perplexity with 160-1,300× fewer calibration tokens and 11-59× fewer GPU hours. This indicates that precise initialization and hierarchical reconstruction can approximate the performance gains of end-to-end training without extensive data exposure.

Inference Performance

Custom CUDA kernels were implemented for binary GEMV (token-by-token decoding) and GEMM (batch processing) operations. NanoQuant shows superior performance over both full-precision and state-of-the-art vector quantization methods, QTIP and AQLM.

We also benchmark our custom CUDA kernels with Gemlite [9], highly optimized 1-bit Triton kernels. We find that our kernels outperform Gemlite on 1 NVIDIA H100 GPU, and the gap is more pronounced for consumer GPUs.

5. Discussion

NanoQuant addresses a critical gap in weight quantization. Existing binary PTQ methods suffer from metadata overhead, limiting effective compression to 2-3 bits. QAT methods achieve genuine 1-bit compression but require prohibitive compute. NanoQuant shows that lower-precision PTQ can be achieved with algorithmic advancements.

Notably, limitations include reliance on small calibration datasets, which may degrade performance on distribution shifts. Extending to mixed-precision compression and adaptive per-layer rank selection remain open research directions. Finally, reasoning performance may benefit from expanded calibration datasets, though this trades off computational efficiency.

6. Conclusion

NanoQuant achieves 1-bit and sub-1-bit post-training quantization of large language models through low-rank binary factorization and precise initialization via ADMM. The method compresses a 70B-parameter model to 5.75 GB (24×) in 13 GPU hours, enabling inference on consumer 8GB GPUs.

These results demonstrate that carefully designed post-training methods can approach the compression-accuracy tradeoffs of quantization-aware training while remaining practical for resource-constrained settings. The approach may inform future work in extreme model compression and efficient inference on edge devices.