Distill Once, Adapt Life-Long: Exploring Dataset Distillation for Continual Test-Time Adaptation

ECCV 2026

Hyun-Kurl Jang*, Jihun Kim*, Hyeokjun Kweon*, Kuk-Jin Yoon

* denotes equal contribution

Official code for DO-ALL, a plug-and-play module for Continual Test-Time Adaptation (CTTA) that revisits source information through Dataset Distillation (DD).

Before deployment, DO-ALL distills the source dataset into a compact set of synthetic anchors. During adaptation, each target sample is matched to its nearest anchor, which guides the update via anchor replay, feature alignment, and harm-adaptive blending that rewinds unstable parameter groups toward the source model. DO-ALL plugs into any base CTTA method without changing its objective.

Installation

conda create -n doall python=3.10 -y && conda activate doall
pip install torch==2.3.1 torchvision==0.18.1   # match your CUDA build
pip install -r requirements.txt

Data

Dataset	Download	Expected location
CIFAR10-C / CIFAR100-C	Auto-downloaded on first run	./data/ (managed automatically)
ImageNet-C	ImageNet-C	./data/ImageNet-C/<corruption>/<severity>/...
CCC	CCC	./data/CCC

Pre-trained model weights

All backbones are the standard public checkpoints used by the upstream benchmark and are auto-downloaded on first run — no manual setup:

Benchmark	Backbone	Weights (MODEL.ARCH)	Source
ImageNet-C / CCC	ResNet-50	resnet50, IMAGENET1K_V1	torchvision
CIFAR10-C	WRN-28-10	Standard	RobustBench
CIFAR100-C	ResNeXt-29	Hendrycks2020AugMix_ResNeXt	RobustBench

DD source anchors

The distilled source anchors are released on Google Drive:

Download — unpack into ./DD_anchor/.

Each set is named {dataset}_{DDmethod}_{backbone}. Every distilled image carries its source soft label as a sibling .pt. Point --synpath at an IPC folder whose {dataset}_{backbone} matches the benchmark you run.

Anchor set	Benchmark	DD method
imagenet_WMDD_resnet50/IPC_10	ImageNet-C / CCC (ResNet-50)	WMDD
imagenet_SRe2L_resnet50/IPC_10	ImageNet-C / CCC (ResNet-50)	SRe²L
imagenet_DELT_resnet50/IPC_10	ImageNet-C / CCC (ResNet-50)	DELT
cifar100_WMDD_resnext/IPC_10	CIFAR100-C (ResNeXt-29)	WMDD
cifar100_SRe2L_resnext/IPC_10	CIFAR100-C (ResNeXt-29)	SRe²L
cifar100_DELT_resnext/IPC_10	CIFAR100-C (ResNeXt-29)	DELT

Example: --synpath ./DD_anchor/imagenet_WMDD_resnet50/IPC_10.

Quick start

# baseline (upstream CTTA method)
python test_time.py --cfg cfgs/imagenet_c/roid.yaml

# the SAME baseline + DO-ALL — one flag
python test_time.py --cfg cfgs/imagenet_c/roid.yaml \
    --do_all --synpath ./DD_anchor/imagenet_WMDD_resnet50/IPC_10

Flag	Meaning
--do_all	Enable the DO-ALL plug-in (sets cfg.DOALL.ENABLED).
--synpath <root>/IPC_<N>	DD anchor ImageFolder (N = images-per-class). Required with --do_all.
--stride k	Run the anchor branch every k steps for cheaper adaptation.

Reproduce Table 2 (ImageNet-to-ImageNet-C, ResNet-50, severity 5, continual)

SYN_ROOT=./DD_anchor/imagenet_WMDD_resnet50 GPU=0 bash run_table2.sh
# runs the 3 baselines + their 3 +DO-ALL variants; writes run_logs/<method>.log

Acknowledgements

Built on mariodoebler/test-time-adaptation (EATA, ROID, RMT, and RobustBench utilities). We thank the authors for releasing their code.

Citation

If you find this work useful, please cite:

@article{jang2026distill,
  title={Distill Once, Adapt Life-Long: Exploring Dataset Distillation for Continual Test-Time Adaptation},
  author={Jang, Hyun-Kurl and Kim, Jihun and Kweon, Hyeokjun and Yoon, Kuk-Jin},
  journal={arXiv preprint arXiv:2606.20196},
  year={2026}
}