README.md

December 13, 2024 ยท View on GitHub

Agony Aunt App

  • This LLM and RAG app was built using Modal Labs, vLLM and FastHTML.
  • Try it out yourself with my demo here
  • Blog post for more details here
  • Download database data from volume modal volume get db_data /chat_history.db . inspect with the ipynb notebook.
  • Deploy app on modal: modal deploy app_name.py.
  • Download the model and create a faiss index: modal run script_name.py.
  • chat_advan_v2_buffer.py is the most recent version, incoporates a buffer system for cleaner LLM output stream. However, edge case still occur where spacing is sometimes ommited.