README.md

Installation

conda create --name lavida python=3.13
conda activate lavida
pip install -e .[train]
cd eval
pip install -e .
cd ../
pip install trl==0.17.0 

Download Checkpoint

Please download checkpoints from Huggingface and organize them in the following structure

<repo root>
--lavida-ckpts # create this folder via mkdir
   --lavida-llada-hd # jacklishufan/lavida-llada-v1.0-instruct

Inference

run the following script to perfom standard inference and text-infilling

python predict.py
python predict_fim.py

Evaluation

Reproduce Main Evaluation Results

bash eval/run.sh lavida-ckpts/lavida-llada-hd

Modify the retaining ratio

Change the ratio at line 304 in RedVTP-LaViDa/llava/model/language_model/llada/generate.py:

keep_ratio = 0.50