README.md

• 🔗 Pipeline-as-YAML - chain ingest, filter, transform, and LLM-assess stages declaratively
• ⚡ Parallel execution - ThreadPoolExecutor with configurable concurrency per stage
• 💾 Resumable runs - atomic per-sample state on disk; Ctrl+C and re-run to pick up where you left off
• 🤖 LLM integration - Jinja2 prompt templates with any LLM provider via LiteLLM
• 🌐 REST API (WIP) - query run state and sample data over HTTP (currently experimental and incomplete)
• 🧩 Extensible - write custom processors as Python classes, reference them by path in YAML

---

📚 Documentation

The documentation covers architecture, pipeline schemas, CLI references, and custom processor development.

👉 Read the Official Documentation here

---

⚡️ Quickstart

Try a complete local run with the included text samples. No API keys, Ghidra, or driver corpus needed. Requires Python 3.11+.

git clone https://github.com/416rehman/DeepZero.git
cd DeepZero
python -m pip install -e .
deepzero run pipelines/demo/samples -p pipelines/demo/pipeline.yaml
deepzero report -p pipelines/demo/pipeline.yaml --open

These commands work in PowerShell and POSIX shells. For an isolated installation, create and activate a Python virtual environment before installing.

The demo discovers two harmless text files, keeps one, filters the smaller one, and generates a browsable HTML report. Run the same pipeline command again to resume from saved state. This demonstrates the engine; it does not run vulnerability analysis. See the demo walkthrough for expected results and configuration experiments.

For the driver analysis pipeline, follow the full setup guide and pipeline prerequisites. Optional integrations require their own dependencies and configuration.

If DeepZero is useful to your work, star this repository to help others discover it. Feedback on your first run is welcome in the issue tracker.

---

📁 Repository Structure

src/deepzero/
├── api/                 # REST API (starlette)
├── engine/              # orchestration, state persistence, pipeline execution
└── stages/              # built-in processors (map, reduce, ingest)

processors/              # external processors (shipped as examples)
├── ghidra_decompile/    # ghidra headless decompiler (MapProcessor)
├── loldrivers_filter/   # loldrivers.io hash exclusion filter (MapProcessor)
├── pe_ingest/           # PE header parser and driver metadata extractor (IngestProcessor)
└── semgrep_scanner/     # semgrep batch scanner (BulkMapProcessor)

pipelines/
├── demo/                # local first run with harmless text files; no API keys
└── loldrivers/          # BYOVD kernel driver vulnerability research pipeline
    ├── pipeline.yaml
    ├── assessment.j2    # LLM prompt template
    └── rules/           # semgrep rules

docs/                    # Jekyll-based GitHub Pages documentation
tests/                   # pytest suite

---

🤝 Contributing

CI runs on Python 3.11, 3.12, 3.13, and 3.14 via GitHub Actions.

Run linting and security checks before submitting:

ruff check . && ruff format --check . && bandit -ll -ii -c pyproject.toml -r .

Please refer to the Contributing Guide and the Code of Conduct before submitting pull requests.

---

📄 License

DeepZero is released under the MIT License.