Red Team Example: Garak Scanner on LLM Sandbox

This directory contains a complete, end‑to‑end example of using Garak, an LLM vulnerability scanner, against a local LLM sandbox.

The setup uses garak to probe the llm_local sandbox via its Gradio interface (port 7860), simulating a red team operation to find vulnerabilities like prompt injection, hallucination, and more.

---

📋 Table of Contents

1. Attack Strategy
2. Prerequisites
3. Running the Sandbox
4. Configuration
5. Attack Workflow
6. Cleaning Up
7. Files Overview
8. OWASP Top 10 Coverage

---

Attack Strategy

graph LR
    subgraph "Attacker Environment (Local)"
        AttackScript[attack.py]
        Config[Garak Config<br/>config/garak.yaml]
        Reports[Reports<br/>reports/]
    end

    subgraph "Garak System"
        GarakCore[Garak Core]
        Generator[Custom Generator<br/>integrate.py]
    end

    subgraph "Target Sandbox (Container)"
        Gradio[Gradio Interface<br/>:7860]
        MockAPI[Mock API Gateway<br/>FastAPI :8000]
        MockLogic[Mock App Logic]
    end

      subgraph "LLM Backend (Local Host)"
        Ollama[Ollama Server<br/>:11434]
        Model[gpt‑oss:20b Model]
    end

    %% Interaction flow
    Config --> AttackScript
    AttackScript --> GarakCore
    GarakCore --> Generator
    Generator -->|Gradio Client| Gradio
    Gradio -->|HTTP POST /v1/chat/completions| MockAPI
    MockAPI --> MockLogic
    MockLogic -->|HTTP| Ollama
    Ollama --> Model
    Model --> Ollama
    Ollama -->|Response| MockLogic
    MockLogic --> MockAPI
    MockAPI -->|Response| Gradio
    Gradio -->|Response| Generator
    Generator --> GarakCore
    GarakCore --> Reports

    style AttackScript fill:#ffcccc,stroke:#ff0000
    style Config fill:#ffcccc,stroke:#ff0000
    style Reports fill:#ffcccc,stroke:#ff0000
    style GarakCore fill:#c2e0ff,stroke:#0066cc
    style Generator fill:#c2e0ff,stroke:#0066cc
    style Gradio fill:#e1f5fe,stroke:#01579b
    style MockAPI fill:#fff4e1
    style MockLogic fill:#fff4e1
    style Ollama fill:#ffe1f5
    style Model fill:#ffe1f5

🔧 Prerequisites

• Podman (or Docker) – container runtime for the sandbox.
• Make – for running the convenience commands.
• uv – for dependency management.

---

🚀 Running the Sandbox

The Makefile provides a set of high‑level commands that abstract away the low‑level container and Python steps.

Target	What it does	Typical usage
make setup	Builds and starts the local LLM sandbox container.	make setup
make attack	Runs the Garak scan against the sandbox using attack.py.	make attack
make stop	Stops and removes the sandbox container.	make stop
make all	Runs stop → setup → attack → stop in one shot.	make all

---

⚙️ Configuration

config/config.toml

Defines the target sandbox environment.

[target]
sandbox = "llm_local"

config/garak.yaml

This file controls the Garak configuration. It defines which probes to run and how to report results.

run:
  spec:
    include:
      - "probes.exploitation"

plugins:
  target_type: "function"
  target_name: "integrate#generate"

reporting:
  report_prefix: "reports/GenAI-Red-Team"
  taxonomy: "owasp"

• run.spec: Determines which probes are active. Entries must use the probes.<name> prefix (e.g., probes.exploitation). Run uv run garak --list_probes (or with -v for descriptions/tiers) to see available probes.
• target_name: Points to the custom generator in integrate.py.

---

Outputs

Results are saved to reports/ in the following formats:

• .jsonl - Contains all prompts and answers.
• .html - Contains a summary of the findings.

---

Files Overview

• attack.py: Entry point for the scan.
• integrate.py: Custom Garak generator to interface with the sandbox.
• config/garak.yaml: Garak configuration.
• Makefile: Automation commands.

OWASP Top 10 Coverage

The Garak configuration (config/garak.yaml) has been tuned to include probes that map to the OWASP Top 10 for LLM Applications.

OWASP Top 10 Vulnerability	Garak Probe(s)	Description
LLM01: Prompt Injection	ansiescape, continuation, dan, doctor, dra, encoding, fitd, goodside, latentinjection, phrasing, promptinject, sata	Tests for direct injection, jailbreaks, and encoding obfuscation.
LLM02: Insecure Output Handling	ansiescape, av_spam_scanning, exploitation, fitd, packagehallucination, web_injection	Checks for XSS, RCE, and other output handling vulnerabilities.
LLM04: Model Denial of Service	divergence	Tests for resource exhaustion via divergence.
LLM05: Supply Chain Vulnerabilities	ansiescape, fitd, glitch, goodside	Checks for vulnerabilities in third-party components or data.
LLM06: Sensitive Information Disclosure	divergence, donotanswer, exploitation, grandma, leakreplay, web_injection	Checks for leakage of PII or sensitive data.
LLM09: Overreliance	donotanswer, goodside, misleading, packagehallucination, snowball	Tests for hallucination and false information.
LLM10: Model Theft	divergence, leakreplay, topic	Tests for model extraction or theft.