Prompt Categorizer Notes

In-progress notes for a sub-component of a larger agentic-pipeline project: a small classification model that breaks a raw prompt into its constituent elements so each can be routed and parameterized independently downstream.

Problem

Real-world inputs to agentic workflows — particularly voice notes captured as raw transcripts — are rarely clean, single-purpose prompts. A single utterance often interleaves an instruction, the surrounding context the model needs to answer well, and workflow-specific directives meant for a non-LLM consumer (e.g. a human host).

To support graph-based agentic pipelines that parameterize each of these elements as a separate track, the input needs to be decomposed reliably and cheaply at the edge.

Target categories

The classifier separates a prompt into the following constituent elements:

• Prompt — questions or instructions directed at the AI. The thing the model is actually being asked to do.
• Context — surrounding information intended to improve the quality and accuracy of inference. Not a question; supporting material.
• Host instructions — a workflow-specific element (initially scoped to a podcasting use case) providing instructions for human hosts rather than for the AI.

The category set is intentionally small and pipeline-scoped. It is not meant to be a general-purpose taxonomy of utterance types.

Goals

• Train a reliable, small classification model — latency, cost, and local deployability matter more than benchmark scores.
• Operate on dynamic context extraction at the input edge of agentic workflows — particularly voice-note capture where a single transcript contains multiple discrete elements.
• Emit cleanly separated tracks (prompt / context / host-instructions) so that graph-based agentic pipelines can parameterize each element as an independent input.
• Tolerate transcript-style input: disfluencies, run-on phrasing, missing punctuation, and mis-segmented sentences.

Benefits

Deterministic routing of context vs. instruction

LLMs are inconsistent at deciding for themselves what part of an input is background and what part is the actual ask. In practice, explicitly framing material as "this is context, here is the prompt to act on" yields measurably better answers. A categorizer makes that boundary deterministic instead of leaving it to the model to infer — every downstream node in the pipeline gets the same, stable view of what is instruction and what is supporting material.

Decomposition into a prompt array over shared context

A single utterance often contains more than one question. Once context is cleanly separated, the prompt track itself can be further decomposed into an array of discrete questions or sub-instructions, each fanned out against the same shared context. This is a secondary objective for the categorization model and a natural fit for graph-based pipelines that want to parallelize sub-prompts without re-attaching context manually each time.

Differential routing for memory and personalization

The categories have very different long-term value, which lets each track be routed to the storage layer that actually fits it:

• The prompt itself is often ephemeral — a one-off question unlikely to yield ongoing value. If anything, it might be routed to a prompt library for reuse, not into long-term memory.
• The context embedded in the same utterance is frequently the durable part — facts about the user, their projects, preferences, or environment — and is a strong candidate for a persistent memory store backing the application.

Without categorization, both get blended into the same conversation log; with it, you can keep a clean memory store and avoid polluting it with transient questions.

Non-goals (for now)

• A general-purpose intent or dialogue-act classifier.
• Anything that requires a large model or cloud inference at the categorization step.
• Speaker diarization, summarization, or downstream generation — those are separate stages.

Status

Notes-stage. This repository is a working scratchpad for the sub-project; design decisions, dataset notes, and model experiments will be added here as the work progresses.