Haku-Phi: A Phi playground

Haku - 珀

Original model: https://huggingface.co/microsoft/phi-1_5

What is this repo

Just a playground to place lot of things about phi. Maybe trainer/inference ui/optimizations/some addon archs...etc

Plans:

• optimizations: like xformers/pytorch sdp attn
• trainer: trainer for LLM/VQA/Image Captioner
• inference ui: Like what I have done on Guanaco-7B (not the Q-LoRA one).
• some addon archs: For VQA or t2i model.