Changsheng Zhao*, Ernie Chang*, Zechun Liu*†, Chia-Jung Chang, Wei Wen, Chen Lai, Rick Cao, Yuandong Tian, Raghuraman Krishnamoorthi, Yangyang Shi, Vikas Chandra
† Corresponding Author and Research Lead
MobileLLM-R1: Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes
Paper  |  Models  |  Code

Zechun Liu, Changsheng Zhao, Hanxian Huang, Sijia Chen, Jing Zhang, Jiawei Zhao, Scott Roy, Lisa Jin, Yunyang Xiong, Yangyang Shi, Lin Xiao, Yuandong Tian, Bilge Soran, Raghuraman Krishnamoorthi, Tijmen Blankevoort, Vikas Chandra
ParetoQ: Scaling Laws in Extremely Low-Bit LLM Quantization
Paper  |  Code  |  Models

Zechun Liu*, Changsheng Zhao*, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, Tijmen Blankevoort
SpinQuant: LLM Quantization with Learned Rotations
Paper |  Code  |  Post  |  Models

Zechun Liu, Changsheng Zhao, Forrest Iandola, Chen Lai, Yuandong Tian, Igor Fedorov, Yunyang Xiong, Ernie Chang, Yangyang Shi, Raghuraman Krishnamoorthi, Liangzhen Lai, Vikas Chandra
MobileLLM: Optimizing Sub-Billion Parameter Language Models for On-Device Use Cases
Forty-first International Conference on Machine Learning, 2024.
Paper  |  Code  |  Models

Zechun Liu*,Barlas Oguz*, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, Vikas Chandra
LLM-QAT: Data-Free Quantization Aware Training for Large Language Models
ACL 2023 Findings.
Paper  |  Code

Yunyang Xiong, Chong Zhou, Xiaoyu Xiang, Lemeng Wu, Chenchen Zhu, Zechun Liu, Saksham Suri, Balakrishnan Varadarajan, Ramya Akula, Forrest Iandola, Raghuraman Krishnamoorthi, Bilge Soran, Vikas Chandra
Efficient Track Anything
ICCV 2024.
Paper  |  Code  |  Project Page  |  Models

Xiaoqian Shen, Yunyang Xiong, Changsheng Zhao, Lemeng Wu, Jun Chen, Chenchen Zhu, Zechun Liu, Fanyi Xiao, Balakrishnan Varadarajan, Florian Bordes, Zhuang Liu, Hu Xu, Hyunwoo J Kim, Bilge Soran, Raghuraman Krishnamoorthi, Mohamed Elhoseiny, Vikas Chandra
LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding
ICML 2024.
Paper  |  Code  |  Project Page

Mingchen Zhuge, Changsheng Zhao, Dylan Ashley, Wenyi Wang, Dmitrii Khizbullin, Yunyang Xiong, Zechun Liu, Ernie Chang, Raghuraman Krishnamoorthi, Yuandong Tian, Yangyang Shi, Vikas Chandra, Jürgen Schmidhuber
Agent-as-a-Judge: Evaluate Agents with Agents
ICML 2024.
Paper  |  Dataset  |  Code

Lisa Jin, Jianhao Ma, Zechun Liu, Andrey Gromov, Aaron Defazio, Lin Xiao
PARQ: Piecewise-Affine Regularized Quantization
ICML 2024.
Paper  |  Code

Zhenyu Zhang, Zechun Liu, Yuandong Tian, Harshit Khaitan, Zhangyang Wang, Steven Li
R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inference
ICLR 2024.
Paper  |  Code

Shih-yang Liu*, Zechun Liu*, Xijie Huang, Pingcheng Dong, Kwang-Ting Cheng
LLM-FP4: 4-Bit Floating-Point Quantized Transformers
EMNLP 2023.
Paper  |  Code

Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, Mohamed Elhoseiny
MiniGPT-v2: Large Language Model as a Unified Interface for Vision-Language Multi-Task Learning
Paper  |  Project Page

Zechun Liu*, Barlas Oguz*, Aasish Pappu, Yangyang Shi, Raghuraman Krishnamoorthi
Binary and Ternary Natural Language Generation  |  Code ACL 2023 (Oral).
Paper

Zechun Liu*, Barlas Oguz*, Aasish Pappu, Lin Xiao, Scott Yih, Meng Li, Raghuraman Krishnamoorthi, Yashar Mehdad
BiT: Robustly Binarized Multi-Distilled Transformer
Paper  |  Code NeurIPS 2022.

Shih-Yang Liu*, Zechun Liu*, Kwang-Ting Cheng
Oscillation-Free Quantization for Low-Bit Vision Transformers
ICML 2023.
Paper  |  Code

Zechun Liu, Kwang-Ting Cheng, Dong Huang, Eric P Xing, Zhiqiang Shen
Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimation
CVPR 2022.
Paper  |  Code

Zechun Liu*, Zhiqiang Shen*, Shichao Li, Koen Helwegen, Dong Huang, Kwang-Ting Cheng.
How Do Adam and Training Strategies Help BNNs Optimization
International Conference on Machine Learning (ICML), 2021.
Code & Models  |  Paper

Zechun Liu, Zhiqiang Shen, Marios Savvides, Kwang-Ting Cheng.
ReActNet: Towards Precise Binary NeuralNetwork with Generalized Activation Functions
European Conference on Computer Vision (ECCV), 2020.
Code & Models  |  Paper
We achieve 65.9% (ResNet-based) and 69.5% (MobileNet-based) top-1 accuracy on ImageNet (the new results are slightly higher than those in our original paper after we fix a small loading bug), for the first time, exceeding the benchmarking ResNet-level accuracy (69.3%) while achieving more than 22× reduction in computational complexity.

Zichao Guo, Xiangyu Zhang, Haoyuan Mu, Wen Heng, Zechun Liu, Yichen Wei, Jian Sun.
Single Path One-Shot Neural Architecture Search with Uniform Sampling
European Conference on Computer Vision (ECCV), 2020.
Code & Models  |  Paper

Zechun Liu, Haoyuan Mu, Xiangyu Zhang, Zichao Guo, Xin Yang, Kwang-Ting Cheng, Jian Sun.
MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning
IEEE International Conference on Computer Vision (ICCV), 2019.
Code & Models  |  Paper

Zechun Liu, Baoyuan Wu, Wenhan Luo, Xin Yang, Wei Liu, Kwang-Ting Cheng.
Bi-Real Net: Enhancing the Performance of 1-bit CNNs with Improved Representational Capability and Advanced Training Algorithm
European Conference on Computer Vision (ECCV), 2018.
Code & Models  |  Paper

Koen Helwegen, James Widdicombe, Lukas Geiger, Zechun Liu, Kwang-Ting Cheng, Roeland Nusselder.
Latent Weights Do Not Exist: Rethinking Binarized Neural Network Optimization
Advances in Neural Information Processing Systems 32 pre-proceedings (NIPS), 2019.
Code & Models  |  Paper