NVIDIA Data Flywheel Foundational Blueprint Notebooks

Introduction

A data flywheel creates a self-reinforcing loop where user interactions continuously enhance the AI application. As users engage, their input helps identify more efficient models (or NIMs) that achieve comparable accuracy, reducing the total cost of ownership. Additionally, these interactions can help improve or maintain model accuracy, leading to better responses and contribute to the ongoing generation of higher-quality data.

Key steps in a data flywheel include logging user interactions, processing the data, customizing and evaluating candidate models, adding guardrails, and integrating them with external knowledge bases for question answering.

How it Works

The Data Flywheel Blueprint provides a unified API (the Orchestrator) that abstracts away the complexity of directly managing NVIDIA NeMo Microservices. As a user, you interact only with the Data Flywheel Orchestrator API to:

• Launch new flywheel jobs (for fine-tuning, evaluation, and deployment of models)
• Monitor job progress and results
• Access evaluation metrics and customization status

What happens under the hood:

When you submit a job via the Orchestrator API, the system:

• Retrieves and processes your data (e.g., from Elasticsearch)
• Creates and uploads datasets for training, evaluation, and validation
• Deploys and manages candidate models (NIMs) on the NeMo platform
• Runs evaluations (including LLM-as-a-judge if configured)
• Handles model customization and tracks progress
• Aggregates results and exposes them through the same API

All direct communication with the NeMo Microservices platform (model deployment, evaluation, customization, etc.) is handled by the orchestrator's backend services. This means you do not need to interact with NeMo APIs or infrastructure directly—the orchestrator manages the full workflow for you.

Notebooks

Check out the following example notebooks to learn how to optimize LLMs using a data flywheel.

• Discover More Cost-Efficient AI Customer Service Agents: Use logs and tool-calling data to fine-tune and evaluate a more cost efficient model.

Prerequisites

Hardware Requirement

To complete this tutorial, you'll need a system with at least two A100 or H100 (80GB) NVIDIA GPUs, which will be used as follows:

• Fine-tuning: At least one GPU is required for fine-tuning a model (e.g.meta/llama-3.2-1B-instruct, meta/llama-3.2-3B-instruct or meta/llama-3.1-8B-instruct).
• Inference: At least one GPU is required for deploying the corresponding NIM for evaluation.

Software Requirement

You will deploy the NVIDIA NeMo Microservices as part of this blueprint.

First, please ensure your platform meets the Requirements before proceeding. The notebook uses a script to automate the remaining setup, including the minikube cluster and NeMo microservices deployment.

Get the Data Flywheel Blueprint

1. Clone the blueprint repository:

   git clone git@github.com:NVIDIA-AI-Blueprints/data-flywheel.git
   
   cd data-flywheel
   

2. Install dependencies using uv:

   uv sync --dev
   

Access the Tutorial

1. Launch Jupyter Lab to begin working with the provided tutorial.

   uv run --with jupyter jupyter lab --ip=0.0.0.0
   

2. Navigate to the notebook.