self-supervised-learning.md

Self-supervised Learning

• (arXiv 2021.03) Can Vision Transformers Learn without Natural Images? [Paper], [Code]
• (arXiv 2021.04) An Empirical Study of Training Self-Supervised Visual Transformers, [Paper]
• (arXiv 2021.04) SiT: Self-supervised vIsion Transformer, [Paper]], [Code]
• (arXiv 2021.04) VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text, [Paper], [Code]
• (arXiv 2021.04) Emerging Properties in Self-Supervised Vision Transformers, [Paper], [Code]
• (arXiv 2021.05) Self-Supervised Learning with Swin Transformers, [Paper], [Code]
• (arXiv 2021.06) MST: Masked Self-Supervised Transformer for Visual Representation, [Paper]
• (arXiv 2021.06) Efficient Self-supervised Vision Transformers for Representation Learning, [Paper]
• (arXiv 2021.09) Localizing Objects with Self-Supervised Transformers and no Labels, [Paper]
• (arXiv 2021.10) Revitalizing CNN Attentions via Transformers in Self-Supervised Visual Representation Learning, [Paper], [Code]
• (arXiv 2022.01) RePre: Improving Self-Supervised Vision Transformer with Reconstructive Pre-training, [Paper], [Code]
• (arXiv 2022.02) Self-Supervised Transformers for Unsupervised Object Discovery using Normalized Cut, [Paper], [Project]
• (arXiv 2022.03) Mugs: A Multi-Granular Self-Supervised Learning Framework, [Paper], [Code]
• (arXiv 2022.04) A Transformer-Based Contrastive Learning Approach for Few-Shot Sign Language Recognition, [Paper]
• (arXiv 2022.04) DILEMMA: Self-Supervised Shape and Texture Learning with Transformers, [Paper]
• (arXiv 2022.04) Self-supervised Vision Transformers for Joint SAR-optical Representation Learning, [Paper]
• (arXiv 2022.05) UTC: A Unified Transformer with Inter-Task Contrastive Learning for Visual Dialog, [Paper]
• (arXiv 2022.05) Uniform Masking: Enabling MAE Pre-training for Pyramid-based Vision Transformers with Locality, [Paper], [Code]
• (arXiv 2022.05) Self-Supervised Pre-training of Vision Transformers for Dense Prediction Tasks, [Paper], [Code]
• (arXiv 2022.05) A Closer Look at Self-supervised Lightweight Vision Transformers, [Paper]
• (arXiv 2022.06) Where are my Neighbors? Exploiting Patches Relations in Self-Supervised Vision Transformer, [Paper], [Code]
• (arXiv 2022.06) Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning, [Paper], [Code]
• (arXiv 2022.06) Exploring Feature Self-relation for Self-supervised Transformer, [Paper]
• (arXiv 2022.06) Position Labels for Self-Supervised Vision Transformer, [Paper]
• (arXiv 2022.06) Adapting Self-Supervised Vision Transformers by Probing Attention-Conditioned Masking Consistency, [Paper], [Code]
• (arXiv 2022.06) Patch-level Representation Learning for Self-supervised Vision Transformers, [Paper], [Code]
• (arXiv 2022.07) Hierarchically Self-Supervised Transformer for Human Skeleton Representation Learning, [Paper], [Code]
• (arXiv 2022.08) Self-Supervised Vision Transformers for Malware Detection, [Paper]
• (arXiv 2022.09) Prior Knowledge-Guided Attention in Self-Supervised Vision Transformers, [Paper]
• (arXiv 2022.10) Attention Distillation: self-supervised vision transformer students need more guidance, [Paper], [Code]
• (arXiv 2022.10) Histopathological Image Classification based on Self-Supervised Vision Transformer and Weak Labels, [Paper], [Code]
• (arXiv 2022.10) Learning Self-Regularized Adversarial Views for Self-Supervised Vision Transformers, [Paper], [Code]
• (arXiv 2022.10) SSiT: Saliency-guided Self-supervised Image Transformer for Diabetic Retinopathy Grading, [Paper], [Code]
• (arXiv 2022.10) PatchRot: A Self-Supervised Technique for Training Vision Transformers, [Paper], [Code]
• (arXiv 2022.10) Foreign Object Debris Detection for Airport Pavement Images based on Self-supervised Localization and Vision Transformer, [Paper]
• (arXiv 2022.12) Location-Aware Self-Supervised Transformers, [Paper], [Code]
• (arXiv 2023.02) Real Estate Property Valuation using Self-Supervised Vision Transformers, [Paper]
• (arXiv 2023.02) Layer Grafted Pre-training: Bridging Contrastive Learning And Masked Image Modeling For Label-Efficient Representations, [Paper], [Code]
• (arXiv 2023.03) ST-KeyS: Self-Supervised Transformer for Keyword Spotting in Historical Handwritten Documents, [Paper]
• (arXiv 2023.03) AdPE: Adversarial Positional Embeddings for Pretraining Vision Transformers via MAE+, [Paper], [Code]
• (arXiv 2023.03) Contrastive Transformer: Contrastive Learning Scheme with Transformer innate Patches, [Paper]
• (arXiv 2023.04) Token Boosting for Robust Self-Supervised Visual Transformer Pre-training, [Paper]
• (arXiv 2023.04) MOST: Multiple Object localization with Self-supervised Transformers for object discovery, [Paper]
• (arXiv 2023.05) LostPaw: Finding Lost Pets using a Contrastive Learning-based Transformer with Visual Input, [Paper]
• (arXiv 2023.05) What Do Self-Supervised Vision Transformers Learn, [Paper], [Code]
• (arXiv 2023.06) Improving Visual Prompt Tuning for Self-supervised Vision Transformers, [Paper], [Code]
• (arXiv 2023.06) DenseDINO: Boosting Dense Self-Supervised Learning with Token-Based Point-Level Consistency, [Paper]
• (arXiv 2023.07) Distilling Self-Supervised Vision Transformers for Weakly-Supervised Few-Shot Classification & Segmentation, [Paper]
• (arXiv 2023.07) Self-supervised Representation Learning by Predicting Masked Online Codebook Assignments, [Paper]
• (arXiv 2023.08) Emergence of Segmentation with Minimalistic White-Box Transformers, [Paper], [Code]
• (arXiv 2023.10) Limited Data, Unlimited Potential:A Study on ViTs Augmented by Masked Autoencoders, [Paper], [Code]
• (arXiv 2023.11) LISBET: a self-supervised Transformer model for the automatic segmentation of social behavior motifs, [Paper], [Code]
• (arXiv 2024.01) Analyzing Local Representations of Self-supervised Vision Transformers, [Paper]
• (arXiv 2024.04) Parameter Efficient Fine-tuning of Self-supervised ViTs without Catastrophic Forgetting, [Paper], [Code]
• (arXiv 2024.05) S3Former: Self-supervised High-resolution Transformer for Solar PV Profiling, [Paper]
• (arXiv 2024.05) Visualizing the loss landscape of Self-supervised Vision Transformer, [Paper]
• (arXiv 2024.06) Self-Supervised Vision Transformer for Enhanced Virtual Clothes Try-On, [Paper]
• (arXiv 2024.07) Self-supervised Vision Transformer are Scalable Generative Models for Domain Generalization, [Paper], [Code]
• (arXiv 2024.07) Multi-Label Plant Species Classification with Self-Supervised Vision Transformers, [Paper], [Code]
• (arXiv 2024.07) TreeSBA: Tree-Transformer for Self-Supervised Sequential Brick Assembly, [Paper], [Code]
• (arXiv 2024.07) Self-supervised transformer-based pre-training method with General Plant Infection dataset, [Paper], [Code]
• (arXiv 2025.06) Boosting Generative Adversarial Transferability with Self-supervised Vision Transformer Features, [Paper], [Code]
• (arXiv 2025.07) Self-supervised pretraining of vision transformers for animal behavioral analysis and neural encoding, [Paper]
• (arXiv 2025.09) Which Direction to Choose? An Analysis on the Representation Power of Self-Supervised ViTs in Downstream Tasks, [Paper]
• (arXiv 2025.09) FHRFormer: A Self-supervised Transformer Approach for Fetal Heart Rate Inpainting and Forecasting, [Paper]