Post-training quantization compresses a neural network within few hours with only a small unlabeled calibration set. However, so far it has been only discussed and empirically demonstrated in the context of uniform quantization on convolutional neural networks. We thus propose a new post-training {\it non-uniform} quantization method, called Mr.BiQ, allowing low bit-width quantization even on {\it Transformer} models. In particular, we leverage the binary coding for weights while allowing activations to be represented as various data formats (e.g., INT8, bfloat16, and FP32) in addition to the binary coding. Unlike conventional methods which optimize full-precision weights first, then decompose the weights into quantization parameters, Mr.BiQ recognizes the quantization parameters (i.e., scaling factors and bit-code) as directly and simultaneously learnable parameters during the optimization. To verify the superiority of the proposed quantization scheme, we test Mr.BiQ on various models including convolutional neural networks and Transformer models.  According to experimental results, Mr.BiQ shows significant improvement in terms of accuracy when the bit-width of weights is equal to 2.

https://openaccess.thecvf.com/content/CVPR2022/html/Jeon_Mr.BiQ_Post-Training_Non-Uniform_Quantization_Based_on_Minimizing_the_Reconstruction_Error_CVPR_2022_paper.html