Workshop Quick Start Guide

Prerequisites

1. Install Foundry Local

Follow the official installation guide: https://github.com/microsoft/Foundry-Local

# Start Foundry Local service
foundry service start

# Load a model (phi-4-mini recommended for workshop)
foundry model run phi-4-mini

# Verify service is running
foundry service status

2. Install Python Dependencies

From the Workshop directory:

# Create virtual environment (recommended)
python -m venv .venv

# Activate virtual environment
# Windows:
.venv\Scripts\activate
# macOS/Linux:
source .venv/bin/activate

# Install requirements
pip install -r requirements.txt

Running Workshop Samples

Session 01: Basic Chat

cd Workshop/samples
python -m session01.chat_bootstrap "What are the benefits of local AI?"

Environment Variables:

set FOUNDRY_LOCAL_ALIAS=phi-4-mini
set SHOW_USAGE=1

Session 02: RAG Pipeline

cd Workshop/samples
python -m session02.rag_pipeline

Environment Variables:

set FOUNDRY_LOCAL_ALIAS=phi-4-mini
set RAG_QUESTION="Why use RAG with local inference?"
set EMBED_MODEL=sentence-transformers/all-MiniLM-L6-v2

Session 02: RAG Evaluation (Ragas)

cd Workshop/samples
python -m session02.rag_eval_ragas

Note: Requires additional dependencies installed via requirements.txt

Session 03: Benchmarking

cd Workshop/samples
python -m session03.benchmark_oss_models

Environment Variables:

set BENCH_MODELS=phi-4-mini,qwen2.5-0.5b
set BENCH_ROUNDS=5
set BENCH_PROMPT="Explain RAG briefly"
set BENCH_STREAM=1

Output: JSON with latency, throughput, and first-token metrics

Session 04: Model Comparison

cd Workshop/samples
python -m session04.model_compare

Environment Variables:

set SLM_ALIAS=phi-4-mini
set LLM_ALIAS=qwen2.5-7b
set COMPARE_PROMPT="List 5 benefits of local AI inference"

Session 05: Multi-Agent Orchestration

cd Workshop/samples
python -m session05.agents_orchestrator

Environment Variables:

set AGENT_MODEL_PRIMARY=phi-4-mini
set AGENT_MODEL_EDITOR=phi-4-mini
set AGENT_QUESTION="Explain why edge AI matters for compliance"

Session 06: Model Router

cd Workshop/samples
python -m session06.models_router

Tests routing logic with multiple intents (code, summarize, classification)

Session 06: Pipeline

python -m session06.models_pipeline

Complex multi-step pipeline with planning, execution, and refinement

Scripts

Export Benchmark Report

cd Workshop/scripts
python export_benchmark_markdown.py \
    --models "phi-4-mini,qwen2.5-0.5b" \
    --prompt "Explain RAG briefly" \
    --rounds 3 \
    --output benchmark_report.md

Output: Markdown table + JSON metrics

Lint Markdown CLI Patterns

python lint_markdown_cli.py --verbose

Purpose: Detect deprecated CLI patterns in documentation

Testing

Smoke Tests

cd Workshop
python -m tests.smoke

Tests: Basic functionality of key samples

Troubleshooting

Service Not Running

# Check status
foundry service status

# Start if not running
foundry service start

# Load a model
foundry model run phi-4-mini

Module Import Errors

# Ensure virtual environment is activated
.venv\Scripts\activate  # Windows
source .venv/bin/activate  # macOS/Linux

# Reinstall dependencies
pip install -r requirements.txt

Connection Errors

# Check endpoint
foundry service status

# Set explicit endpoint if needed
set FOUNDRY_LOCAL_ENDPOINT=http://localhost:8000

Model Not Found

# List available models
foundry model list

# Download and run a model
foundry model run phi-4-mini

Environment Variable Reference

Core Configuration

Variable	Default	Description
FOUNDRY_LOCAL_ALIAS	Varies	Model alias to use
FOUNDRY_LOCAL_ENDPOINT	Auto	Override service endpoint
SHOW_USAGE	0	Show token usage stats
RETRY_ON_FAIL	1	Enable retry logic
RETRY_BACKOFF	1.0	Initial retry delay (seconds)

Session-Specific

Variable	Default	Description
EMBED_MODEL	sentence-transformers/all-MiniLM-L6-v2	Embedding model
RAG_QUESTION	See sample	RAG test question
BENCH_MODELS	Varies	Comma-separated models
BENCH_ROUNDS	3	Benchmark iterations
BENCH_PROMPT	See sample	Benchmark prompt
BENCH_STREAM	0	Measure first-token latency
AGENT_MODEL_PRIMARY	phi-4-mini	Primary agent model
AGENT_MODEL_EDITOR	Primary	Editor agent model
SLM_ALIAS	phi-4-mini	Small language model
LLM_ALIAS	qwen2.5-7b	Large language model
COMPARE_PROMPT	See sample	Comparison prompt

Recommended Models

Development & Testing

• phi-4-mini - Balanced quality and speed
• qwen2.5-0.5b - Very fast for classification
• gemma-2-2b - Good quality, moderate speed

Production Scenarios

• phi-4-mini - General purpose
• deepseek-coder-1.3b - Code generation
• qwen2.5-7b - High quality responses

SDK Documentation

• Foundry Local: https://github.com/microsoft/Foundry-Local
• Python SDK: https://github.com/microsoft/Foundry-Local/tree/main/sdk/python/foundry_local

Getting Help

1. Check service status: foundry service status
2. View logs: Check Foundry Local service logs
3. Check SDK docs: https://github.com/microsoft/Foundry-Local
4. Review sample code: All samples have detailed docstrings

Next Steps

1. Complete all workshop sessions in order
2. Experiment with different models
3. Modify samples for your use cases

---

Last Updated: 2025-01-08
Workshop Version: Latest
SDK: Foundry Local Python SDK