Services

Auto-Labeling is built from small, single-purpose services. Each service reads and writes one shared scene format (a DAFT scene — the per-item output directory described in Experiment Output Layout), so services can run standalone, be chained by the workflow runner, or be orchestrated by an external platform.

Every page below follows the same structure: what the service does, what it needs before you start it, a copy-pasteable walkthrough, how to confirm it worked, and where to look when it doesn't.

If this is your first time running Auto-Labeling, start with Getting Started. Come back here when you need one stage on its own. If you are adding a service, see Adding A New Task Or Service.

Index

Service	What it does	Needs a GPU + local checkpoint?	Needs a VLM/LLM endpoint?
Super Resolution	Upscales low-resolution video before annotation	Yes (SeedVR2)	No
Detection and Tracking	Finds and tracks objects across a video or image	Yes (RF-DETR or SAM3)	No
Captioning	Describes what is happening in an image or video	No	Yes (VLM, optional LLM)
Visual QA	Answers a fixed question bank about a scene	No	Yes (VLM, optional LLM)
Reasoning	Derives higher-level events and open-ended QA from existing scene data	No	Yes (LLM)
Visual Attribute Search	Assembles person attributes and generates tiered search queries	No	Yes (LLM, optional VLM upstream)
2D Grounding	Links caption phrases to boxes and masks in an image	Yes (SAM3)	Yes (VLM)
Referring Expressions	Generates a short phrase that uniquely identifies each detected box	No	Yes (VLM)
Training Export	Packages completed scenes into a training-ready dataset	No	No
Workflow Runner	Compiles a cookbook into an ordered chain of the services above and runs them locally	Depends on selected stages	Depends on selected stages

Before running any service for real

Every service can be smoke-tested with no models, GPU, or network access — see each page's first step. Before a real run, make sure you have:

1. Followed Installation so make sync has completed.
2. Downloaded the checkpoints your chosen services need — see Model Provisioning.
3. A reachable VLM and/or LLM endpoint — see VLM and LLM Endpoints.
4. A DataEntry JSONL manifest pointing at your media — see Experiment Output Layout for the shape.

Running one service versus a full workflow

You can invoke any service directly with make run SCRIPT=<service>:main — useful for debugging one stage in isolation. For a real end-to-end pipeline (for example: detect, then caption, then answer questions, then reason), use the workflow runner with a cookbook instead of chaining services by hand; it resolves stage order, mounts, and shared scene paths for you.