hippoTune

A prompt-based continual learning framework built on Mammoth, using multi-layer prompt pools with orthogonal regularization and soft attention selection for Vision Transformers.

Features

• Multi-layer prompt pool across configurable ViT layers
• Orthogonal regularization to reduce prompt redundancy
• Soft attention-based prompt selection with temperature control
• Two operating modes: standard and high-efficiency
• Compatible with class-incremental and task-incremental learning
• Built on the Mammoth continual learning framework

Installation

pip install -r requirements.txt

Optional dependencies (for documentation, etc.):

pip install -r requirements-optional.txt

Quick Start

python main.py \
  --model hippoTune \
  --dataset seq-cifar100-224 \
  --model_config default \
  --n_epochs 5 \
  --lr 0.01 \
  --batch_size 128 \
  --optimizer adam \
  --layer_idx '[0,1,2,3,4,5,6,7,8,9]' \
  --mode 1 \
  --T1 0.01 \
  --T2 0.01 \
  --lambda_orth 1

Key Arguments

Argument	Default	Description
--size	10	Prompt pool size
--length	5	Prompt token length
--layer_idx	—	ViT layers to attach prompts, e.g. '[0,1,2,3]'
--mode	1	1 = standard, 2 = high-efficiency
--T1	0.01	Softmax temperature for key set K1
--T2	0.01	Softmax temperature for key set K2
--lambda_orth	1.0	Weight of orthogonal regularization loss
--clip_grad	1.0	Gradient clipping norm
--pretrained	1	Use ImageNet-pretrained ViT backbone
--head_type	token	Classification head input: token, gap, prompt, token+prompt

Supported Datasets

CIFAR-10, CIFAR-100, CIFAR-10-224, CIFAR-100-224, CUB-200, Cars-196, ImageNet-R, TinyImageNet, EuroSAT-RGB, ISIC, RESISC45, CropDisease, ChestX, MIT-67, CelebA, MNIST, and their resizable (_rs) variants.

Project Structure

models/
  hippoTune.py              # Model entry point and training logic
  hippoTune_utils/
    model.py                # Core model wrapper
    prompt.py               # Prompt pool implementation
    attention.py            # Attention mechanism
    vision_transformer.py   # Modified ViT with prompt support
backbone/
  vit.py                    # Vision Transformer backbone
datasets/
  seq_*.py                  # Sequential dataset wrappers

Citation

If you use hippoTune in your research, please cite:

@article{hippoTune,
  title={},
  author={},
  journal={},
  year={}
}

Acknowledgements

This project is built on Mammoth, a modular continual learning framework by Buzzega et al.

License

MIT