SEE: Continual Fine-tuning with Sequential Ensemble of Experts

This is the code repository for the paper SEE: Continual Fine-tuning with Sequential Ensemble of Experts.

The main results of the paper are generated using SSR. However, this framework does not support the latest models, so our additional experiments with the most recent models are conducted using Llamafactory.

Note: When using LoRA tuning, please edit the related chat template to ensure the EOS token matches the PAD token. Otherwise, generation may not stop correctly. This issue has also been reported in Llamafactory.

Continual Training

The scripts directory contains the training scripts for the latest models. MTL refers to the multi-task learning baseline, and AvgTasks refers to the average single-task learning performance. You can run the scripts SEE.train.10.sh, AvgTask.train.10.sh, and MTL.train.10.sh to reproduce the results of the latest models.

Evaluation

After training, please run the scripts SEE.eval.10.sh, AvgTask.eval.10.sh, and MTL.eval.10.sh to generate predictions. Then for SEE methods, please merge the results through:

python merge_predictions.py \
    --categories "qa,qg,sa,sum,trans" \
    --models "Llama-3.1-8B" \
    --method_name "SEE" \
    --save_dir "./saves"

Then, you should run the evaluate.py to evaluate the generated_predictions.jsonl:

python evaluate.py --input_file path/to/generated_predictions.jsonl