Projects

Use these references for the project process:

• Project video
• Project office hours

To receive a certificate of completion for the course, you need to deliver a project. There are two attempts for that.

Submit your project here

Objective

The goal of this project is to apply the course material. You will build an end-to-end RAG or agent application.

Problem statement

For the project, we ask you to build an end-to-end LLM application. It can be a RAG application, an agent application, or a combination of both.

For that, you need:

• Select a dataset or API-backed data source that you're interested in (see Datasets for examples and ideas)
• Ingest the data into a knowledge base, or connect to the API that provides the data
• Implement the application flow: retrieve context from the data, optionally call tools, build the prompt, and send it to an LLM
• Evaluate the performance of your RAG or agent flow
• Create an interface for the application
• Collect user feedback and monitor your application

Datasets you cannot use

The DataTalks.Club Zoomcamp course FAQ documents are used as the knowledge base in the course modules and homework. They cannot be used for your project. Pick a different corpus or dataset.

Project Documentation

Your project rises or falls with its documentation.

Use these recommendations:

• Write for people who didn't take the course. Explain the problem, the data, and the flow without assuming the reader watched the videos.
• Mention the evaluation criteria in your README. This makes it easier for reviewers to find the relevant parts.
• Add screenshots where they help. For example, show the UI, the monitoring dashboard, or examples of answers from your app.
• Explain how to run the project. Include setup steps, dependencies, configuration, and any environment variables.
• Show how the project works. Add examples of inputs and outputs, common use cases, or a short walkthrough.
• Add a short app preview video if it helps. If you're using Streamlit, you can record it from the app's top-right menu (Streamlit Guide). Then drag and drop the video into the online GitHub editor of your README (Ref).
• Split long documentation into separate files if needed. For example, use setup.md, usage.md, or contributing.md.
• Keep the README up to date when the project changes.

Remember, clear and comprehensive documentation not only helps others but is also a valuable reference for yourself in the future.

Technologies

You don't have to limit yourself to technologies covered in the course.

You can use alternatives as well:

• LLM: OpenAI, Ollama, Groq, AWS Bedrock, etc
• Knowledge base: any text, relational or vector database, including in-memory ones like we implemented in the course or SQLite
• Monitoring: Grafana, Kibana, Streamlit, dash, etc
• Interface: Streamlit, dash, Flask, FastAPI, Django, etc (could be UI or API)
• Ingestion pipeline: Kestra, dlt, Mage, Airflow, Prefect, python scripts, etc

If you use a tool that wasn't covered in the course, explain what that tool does. Also explain how to use it.

If you're not certain about some tools, ask in Slack.

Tips and best practices

Follow these tips while preparing the project:

• It's better to create a separate GitHub repository for your project
• Give your project a meaningful title, e.g. "DataTalksClub Zoomcamp Q&A system" or "Nutrition Facts Chat"

Peer Reviewing

To evaluate the projects, we'll use peer reviewing. This is a good opportunity to learn from each other.

To get points for your project, you need to evaluate 3 projects from your peers. You get 3 extra points for each evaluation.

Review Tips

Use these tips when reviewing a project:

• The reviewer is given a public GitHub repo link and a commit-hash
  • to see the code state of the repo at the provided commit hash, use the following URL:
  • https://github.com/{username}/{repo-name}/tree/{commit-hash}
• It's recommended to clone the repository for the review. To clone the project at the commit hash:

  git clone https://github.com/{username}/{repo-name}.git
  git reset --hard {commit-hash}
  

Evaluation Criteria

Use these criteria to score the project:

• Problem description
  • 0 points: The problem is not described
  • 1 point: The problem is described but briefly or unclearly
  • 2 points: The problem is well-described and it's clear what problem the project solves
• Retrieval flow
  • 0 points: No knowledge base or LLM is used
  • 1 point: No knowledge base is used, and the LLM is queried directly
  • 2 points: Both a knowledge base and an LLM are used in the flow
• Retrieval evaluation
  • 0 points: No evaluation of retrieval is provided
  • 1 point: Only one retrieval approach is evaluated
  • 2 points: Multiple retrieval approaches are evaluated, and the best one is used
• LLM evaluation
  • 0 points: No evaluation of final LLM output is provided
  • 1 point: Only one approach (e.g., one prompt) is evaluated
  • 2 points: Multiple approaches are evaluated, and the best one is used
• Interface
  • 0 points: No way to interact with the application at all
  • 1 point: Command line interface, a script, or a Jupyter notebook
  • 2 points: UI (e.g., Streamlit), web application (e.g., Django), or an API (e.g., built with FastAPI)
• Ingestion pipeline
  • 0 points: No ingestion
  • 1 point: Semi-automated ingestion of the dataset into the knowledge base, e.g., with a Jupyter notebook or a Python script
  • 2 points: Automated ingestion with a special tool (e.g., Kestra, dlt, Airflow, Prefect)
• Monitoring
  • 0 points: No monitoring
  • 1 point: User feedback is collected OR there's a monitoring dashboard
  • 2 points: User feedback is collected and there's a dashboard with at least 5 charts
• Containerization
  • 0 points: No containerization
  • 1 point: Dockerfile is provided for the main application OR there's a docker-compose for the dependencies only
  • 2 points: Everything is in docker-compose
• Reproducibility
  • 0 points: No instructions on how to run the code, the data is missing, or it's unclear how to access it
  • 1 point: Some instructions are provided but are incomplete, OR instructions are clear and complete, the code works, but the data is missing
  • 2 points: Instructions are clear, the dataset is accessible, it's easy to run the code, and it works. The versions for all dependencies are specified.
• Best practices
  • Hybrid search: combining both text and vector search (at least evaluating it) (1 point)
  • Document re-ranking (1 point)
  • User query rewriting (1 point)
• Bonus points (not covered in the course)
  • Deployment to the cloud (2 points)
  • Up to 3 extra bonus points if you want to award for something extra (write in feedback for what)

Project ideas

Datasets

Here are some datasets for your projects and potential things you can do with them

• DTC data:
  • Slack dump: book of the week channel, course channels, career questions, etc
  • DTC website with book of the week archives
  • DTC Podcast: transcripts
• Wikis
  • any subsets of Wikipedia
  • any wiki-like data source
  • notion notes
• Articles
  • Index and answer questions from one or multiple articles
• Transcripts
  • Podcast transcripts
  • YouTube video transcripts
• Books
  • Sci-fi, fiction, or non-fiction books
• Slide Decks and pictures
  • OCR and index slide decks (gpt-4o-mini can do that)
  • Describe and index pictures
• Add more here above this line - send a PR!
• Or just ask ChatGPT (see more ideas here in this example)

You can also generate a dataset with an LLM:

• If you have a dataset but you can't publicly release it, you can generate a similar one with an LLM
• Or you can simply think what kind of a dataset you want to have and generate it

Note that your dataset doesn't have to be in thr Q&A form. Check etc/chunking.md to learn more about chunking.

Cheating and plagiarism

Plagiarism in any form is not allowed.

Examples of plagiarism:

• Taking somebody's else notebooks and projects (in full or partly) and using it for your project
• Re-using your own projects (in full or partly) from other courses and bootcamps
• Re-using your appempt 1 project as attempt 2 if you passed attempt 1
• Re-using your project from the previous iterations of the course

Violating any of this will result in 0 points for this project.

Re-using some parts of the code from the course is allowed.