pruning--quantization.md

Pruning & Quantization

• (arXiv 2021.04) Visual Transformer Pruning, [Paper]
• (arXiv 2021.06) Post-Training Quantization for Vision Transformer, [Paper]
• (arXiv 2021.11) PTQ4ViT: Post-Training Quantization Framework for Vision Transformers, [Paper], [Code]
• (arXiv 2021.11) FQ-ViT: Fully Quantized Vision Transformer without Retraining, [Paper]
• (arXiv 2022.01) Q-ViT: Fully Differentiable Quantization for Vision Transformer, [Paper]
• (arXiv 2022.01) VAQF: Fully Automatic Software-hardware Co-design Framework for Low-bit Vision Transformer, [Paper]
• (arXiv 2022.03) Patch Similarity Aware Data-Free Quantization for Vision Transformers, [Paper]
• (arXiv 2022.03) CP-ViT: Cascade Vision Transformer Pruning via Progressive Sparsity Prediction, [Paper]
• (arXiv 2022.07) I-ViT: Integer-only Quantization for Efficient Vision Transformer Inference, [Paper]
• (arXiv 2022.08) Auto-ViT-Acc: An FPGA-Aware Automatic Acceleration Framework for Vision Transformer with Mixed-Scheme Quantization, [Paper]
• (arXiv 2022.09) PSAQ-ViT V2: Towards Accurate and General Data-Free Quantization for Vision Transformers, [Paper], [Code]
• (arXiv 2022.10) EAPruning: Evolutionary Pruning for Vision Transformers and CNNs, [Paper]
• (arXiv 2022.10) SaiT: Sparse Vision Transformers through Adaptive Token Pruning, [Paper]
• (arXiv 2022.10) Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformer, [Paper], [Code]
• (arXiv 2022.10) oViT: An Accurate Second-Order Pruning Framework for Vision Transformers, [Paper]
• (arXiv 2022.11) CPT-V: A Contrastive Approach to Post-Training Quantization of Vision Transformers, [Paper]
• (arXiv 2022.11) NoisyQuant: Noisy Bias-Enhanced Post-Training Activation Quantization for Vision Transformers, [Paper]
• (arXiv 2022.12) Rethinking the Objectives of Vector-Quantized Tokenizers for Image Synthesis, [Paper], [Code]
• (arXiv 2022.12) RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformers, [Paper]
• (arXiv 2023.02) Oscillation-free Quantization for Low-bit Vision Transformers, [Paper]
• (arXiv 2023.03) Q-HyViT: Post-Training Quantization for Hybrid Vision Transformer with Bridge Block Reconstruction, [Paper], [Code]
• (arXiv 2023.03) Scaled Quantization for the Vision Transformer, [Paper]
• (arXiv 2023.03) Towards Accurate Post-Training Quantization for Vision Transformer, [Paper]
• (arXiv 2023.04) Q-DETR: An Efficient Low-Bit Quantized Detection Transformer, [Paper]
• (arXiv 2023.04) Attention Map Guided Transformer Pruning for Edge Device, [Paper]
• (arXiv 2023.05) Patch-wise Mixed-Precision Quantization of Vision Transformer, [Paper]
• (arXiv 2023.05) Boost Vision Transformer with GPU-Friendly Sparsity and Quantization, [Paper]
• (arXiv 2023.05) Bi-ViT: Pushing the Limit of Vision Transformer Quantization, [Paper]
• (arXiv 2023.07) Variation-aware Vision Transformer Quantization, [Paper], [Code]
• (arXiv 2023.08) Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformers, [Paper], [Code]
• (arXiv 2023.08) Vision Transformer Pruning Via Matrix Decomposition, [Paper]
• (arXiv 2023.09) Transformer-VQ: Linear-Time Transformers via Vector Quantization, [Paper], [Code]
• (arXiv 2023.10) LLM-FP4: 4-Bit Floating-Point Quantized Transformers, [Paper], [Code]
• (arXiv 2023.12) QuantAttack: Exploiting Dynamic Quantization to Attack Vision Transformers, [Paper]
• (arXiv 2024.01) LRP-QViT: Mixed-Precision Vision Transformer Quantization via Layer-wise Relevance Propagation, [Paper]
• (arXiv 2024.01) MPTQ-ViT: Mixed-Precision Post-Training Quantization for Vision Transformer, [Paper]
• (arXiv 2024.03) Accelerating ViT Inference on FPGA through Static and Dynamic Pruning, [Paper]
• (arXiv 2024.04) Instance-Aware Group Quantization for Vision Transformers, [Paper], [Code]
• (arXiv 2024.04) Data-independent Module-aware Pruning for Hierarchical Vision Transformers, [Paper], [Code]
• (arXiv 2024.05) Model Quantization and Hardware Acceleration for Vision Transformers: A Comprehensive Survey, [Paper], [Code]
• (arXiv 2024.05) Trio-ViT: Post-Training Quantization and Acceleration for Softmax-Free Efficient Vision Transformer, [Paper]
• (arXiv 2024.05) MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantization, [Paper]
• (arXiv 2024.06) ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation, [Paper], [Code]
• (arXiv 2024.06) MGRQ: Post-Training Quantization For Vision Transformer With Mixed Granularity Reconstruction, [Paper]
• (arXiv 2024.06) An Analysis on Quantizing Diffusion Transformers, [Paper]
• (arXiv 2024.06) ViT-1.58b: Mobile Vision Transformers in the 1-bit Era, [Paper]
• (arXiv 2024.06) Q-DiT: Accurate Post-Training Quantization for Diffusion Transformers, [Paper], [Code]
• (arXiv 2024.07) ADFQ-ViT: Activation-Distribution-Friendly Post-Training Quantization for Vision Transformers, [Paper]
• (arXiv 2024.07) LPViT: Low-Power Semi-structured Pruning for Vision Transformers, [Paper]
• (arXiv 2024.07) Isomorphic Pruning for Vision Models, [Paper], [Code]
• (arXiv 2024.07) LayerShuffle: Enhancing Robustness in Vision Transformers by Randomizing Layer Execution Order, [Paper]
• (arXiv 2024.07) Fisher-aware Quantization for DETR Detectors with Critical-category Objectives, [Paper]
• (arXiv 2024.07) PRANCE: Joint Token-Optimization and Structural Channel-Pruning for Adaptive ViT Inference, [Paper], [Code]
• (arXiv 2024.07) CLAMP-ViT: Contrastive Data-Free Learning for Adaptive Post-Training Quantization of ViTs, [Paper], [Code]
• (arXiv 2024.07) Reducing Vision Transformer Latency on Edge Devices via GPU Tail Effect and Training-free Token Pruning, [Paper]
• (arXiv 2024.07) ERQ: Error Reduction for Post-Training Quantization of Vision Transformers, [Paper]
• (arXiv 2024.07) AdaLog: Post-Training Quantization for Vision Transformers with Adaptive Logarithm Quantizer, [Paper], [Code]
• (arXiv 2024.07) Mixed Non-linear Quantization for Vision Transformers, [Paper], [Code]
• (arXiv 2024.07) MimiQ: Low-Bit Data-Free Quantization of Vision Transformers, [Paper]
• (arXiv 2024.08) DopQ-ViT: Towards Distribution-Friendly and Outlier-Aware Post-Training Quantization for Vision Transformers, [Paper]
• (arXiv 2024.08) Task-Aware Dynamic Transformer for Efficient Arbitrary-Scale Image Super-Resolution, [Paper]
• (arXiv 2024.08) PQV-Mobile: A Combined Pruning and Quantization Toolkit to Optimize Vision Transformers for Mobile Applications, [Paper], [Code]
• (arXiv 2024.08) Pruning By Explaining Revisited: Optimizing Attribution Methods to Prune CNNs and Transformers, [Paper], [Code]
• (arXiv 2024.08) VQ4DiT: Efficient Post-Training Vector Quantization for Diffusion Transformers, [Paper]
• (arXiv 2024.10) ED-ViT: Splitting Vision Transformer for Distributed Inference on Edge Devices, [Paper]
• (arXiv 2024.10) Token Pruning using a Lightweight Background Aware Vision Transformer, [Paper]
• (arXiv 2024.12) Progressive Fine-to-Coarse Reconstruction for Accurate Low-Bit Post-Training Quantization in Vision Transformers, [Paper]
• (arXiv 2024.12) Semantics Prompting Data-Free Quantization for Low-Bit Vision Transformers, [Paper]
• (arXiv 2024.12) MBQ: Modality-Balanced Quantization for Large Vision-Language Models, [Paper], [Code]
• (arXiv 2025.01) Mix-QViT: Mixed-Precision Vision Transformer Quantization Driven by Layer Importance and Quantization Sensitivity, [Paper]
• (arXiv 2025.02) AIQViT: Architecture-Informed Post-Training Quantization for Vision Transformers, [Paper]
• (arXiv 2025.02) Hardware-Friendly Static Quantization Method for Video Diffusion Transformers, [Paper]
• (arXiv 2025.03) Oscillation-Reduced MXFP4 Training for Vision Transformers, [Paper], [Code]
• (arXiv 2025.03) FP4DiT: Towards Effective Floating Point Quantization for Diffusion Transformers, [Paper], [Code]
• (arXiv 2025.04) APHQ-ViT: Post-Training Quantization with Average Perturbation Hessian Based Reconstruction for Vision Transformers, [Paper], [Code]
• (arXiv 2025.04) The Effects of Grouped Structural Global Pruning of Vision Transformers on Domain Generalisation, [Paper]
• (arXiv 2025.04) NuWa: Deriving Lightweight Task-Specific Vision Transformers for Edge Devices, [Paper], [Code]
• (arXiv 2025.06) Q-VDiT: Towards Accurate Quantization and Distillation of Video-Generation Diffusion Transformers, [Paper], [Code]
• (arXiv 2025.06) GPLQ: A General, Practical, and Lightning QAT Method for Vision Transformers, [Paper]
• (arXiv 2025.06) FIMA-Q: Post-Training Quantization for Vision Transformers by Fisher Information Matrix Approximation, [Paper], [Code]
• (arXiv 2025.06) EfficientQuant: An Efficient Post-Training Quantization for CNN-Transformer Hybrid Models on Edge Devices, [Paper]
• (arXiv 2025.07) DFQ-ViT: Data-Free Quantization for Vision Transformers without Fine-tuning, [Paper]
• (arXiv 2025.07) Revisiting Activation Sparsity for Vision Transformers from a Mixed-Precision Quantization Perspective, [Paper]
• (arXiv 2025.07) Patch Pruning Strategy Based on Robust Statistical Measures of Attention Weight Diversity in Vision Transformers, [Paper]
• (arXiv 2025.08) LRQ-DiT: Log-Rotation Post-Training Quantization of Diffusion Transformers for Text-to-Image Generation, [Paper]
• (arXiv 2025.09) Quantized Visual Geometry Grounded Transformer, [Paper], [Code]
• (arXiv 2025.09) CLQ: Cross-Layer Guided Orthogonal-based Quantization for Diffusion Transformers, [Paper],[Code]