Mockability Architecture Audit

February 24, 2026 · View on GitHub

Central analysis of how testable each layer of the vidpipe codebase is, where the mock boundaries live, and what patterns already work well.


1. Architecture Hierarchy Diagram

CLI Entry (src/index.ts)
  └── Pipeline Orchestrator (src/pipeline.ts)

       ├── Agents (src/agents/*)
       │    ├── BaseAgent ← LLMProvider (constructor-injected)
       │    ├── ShortsAgent, MediumVideoAgent, SummaryAgent, BlogAgent, …
       │    └── ProducerAgent (extends BaseAgent + uses tools)
       │         └── LLM Providers (src/providers/*)
       │              ├── CopilotProvider  ── @github/copilot-sdk
       │              ├── OpenAIProvider   ── openai SDK
       │              └── ClaudeProvider   ── @anthropic-ai/sdk

       ├── Stages (src/stages/*)
       │    └── visualEnhancement.ts → Gemini + GraphicsAgent + FFmpeg

       ├── Tools (src/tools/*)
       │    ├── FFmpeg Operations (src/tools/ffmpeg/*)
       │    │    ├── silenceDetection    ── fluent-ffmpeg
       │    │    ├── singlePassEdit      ── execFileRaw (pure buildFilterComplex)
       │    │    ├── captionBurning      ── execFileRaw + temp dirs
       │    │    ├── clipExtraction      ── execFileRaw + ffprobe
       │    │    ├── aspectRatio         ── execFileRaw + face detection
       │    │    ├── audioExtraction     ── fluent-ffmpeg
       │    │    ├── frameCapture        ── fluent-ffmpeg
       │    │    ├── faceDetection       ── ONNX + sharp + ffmpeg
       │    │    └── overlayCompositing  ── execFileRaw
       │    ├── Gemini Client (src/tools/gemini/geminiClient.ts)
       │    │    └── @google/genai SDK
       │    ├── Whisper Client (src/tools/whisper/whisperClient.ts)
       │    │    └── openai SDK (audio transcription)
       │    ├── Caption Generator (src/tools/captions/captionGenerator.ts)
       │    │    └── Pure functions — no I/O
       │    ├── Image Generation (src/tools/imageGeneration.ts)
       │    │    └── OpenAI DALL-E API + sharp
       │    └── Agent Tools (src/tools/agentTools.ts)
       │         └── ffprobe + fs + DALL-E

       ├── Services (src/services/*)
       │    ├── transcription       ── Whisper client + FFmpeg audio extraction
       │    ├── captionGeneration    ── captionGenerator (pure) + fs writes
       │    ├── costTracker          ── singleton, in-memory accumulator
       │    ├── postStore            ── fs-backed queue (JSON + files)
       │    ├── processingState      ── fs-backed state machine (JSON)
       │    ├── queueBuilder         ── postStore + platformContentStrategy
       │    ├── fileWatcher          ── chokidar (EventEmitter)
       │    ├── gitOperations        ── execCommandSync (shell)
       │    ├── lateApi              ── HTTP client (Late.co API)
       │    ├── scheduler            ── lateApi + scheduleConfig + postStore
       │    ├── scheduleConfig       ── fs read/write (schedule.json)
       │    ├── accountMapping       ── lateApi + fs cache
       │    ├── socialPosting        ── placeholder client (interface-ready)
       │    └── platformContentStrategy ── pure data lookup, no I/O

       └── Config (src/config/*)
            ├── environment.ts   ── getConfig() singleton, reads process.env
            ├── logger.ts        ── re-exports from core/logger.ts
            ├── pricing.ts       ── pure functions + const data
            ├── modelConfig.ts   ── reads process.env + const map
            ├── brand.ts         ── cached JSON read (getBrandConfig singleton)
            └── ffmpegResolver.ts ── re-exports from core/ffmpeg.ts
                 └── Core (src/core/*)
                      ├── fileSystem.ts  ── thin wrappers around Node fs
                      ├── ffmpeg.ts      ── path resolution + fluent-ffmpeg factory
                      ├── logger.ts      ── Winston singleton
                      ├── process.ts     ── execFile / execCommand wrappers
                      ├── paths.ts       ── path helpers (projectRoot, fontsDir, etc.)
                      ├── ai.ts          ── OpenAI client re-export
                      ├── media.ts       ── sharp + onnxruntime re-exports
                      ├── network.ts     ── Readable/fetch re-exports
                      ├── env.ts         ── dotenv loader
                      ├── cli.ts         ── Commander re-export
                      ├── text.ts        ── text helpers
                      └── watcher.ts     ── chokidar re-export

2. Mockability Tier System

Tier 1 — Pure Functions (Score 9–10)

No mocking needed. These take inputs and return outputs with zero side effects.

ModuleScoreNotes
tools/captions/captionGenerator.ts9/10Generates SRT/VTT/ASS strings from Transcript. Only import is types. Tests call directly.
config/pricing.ts10/10calculateTokenCost(), calculatePRUCost(), getModelPricing() — pure math on const data.
pipeline.ts → adjustTranscript()10/10Pure timestamp remapping. Tested directly without any mocks.
tools/ffmpeg/singlePassEdit.ts → buildFilterComplex()10/10Pure string builder for FFmpeg filter graphs. Exported and tested in isolation.
tools/ffmpeg/overlayCompositing.ts → getOverlayPosition()10/10Pure expression builder. No I/O.
services/platformContentStrategy.ts9/10Pure data lookup (getMediaRule, platformAcceptsMedia). Only imports types.

Tier 2 — Interface-Injectable (Score 7–8)

Mock via constructor injection; the code explicitly accepts abstractions.

ModuleScoreNotes
agents/BaseAgent.ts7/10Accepts LLMProvider via constructor (provider ?? getProvider()). Tests can inject a mock provider. Tool handlers are subclass methods — side-effectful, but isolated per agent.
providers/CopilotProvider.ts7/10Implements LLMProvider interface. Sessions are created per-call, testable with mock SDK.
providers/OpenAIProvider.ts7/10Same LLMProvider contract. Wraps OpenAI SDK — vi.mock('openai') at module level.
providers/ClaudeProvider.ts7/10Same pattern. vi.mock('@anthropic-ai/sdk').
services/socialPosting.ts8/10Defines SocialPlatformClient interface with post() and validate(). PlaceholderPlatformClient is a test-friendly no-op.

Tier 3 — Module-Mockable (Score 4–6)

Requires vi.mock() at module level. Functions read config, call external processes, or hit APIs, but can be fully mocked with Vitest's ESM mock system.

ModuleScoreNotes
pipeline.ts → processVideo()5/10Orchestrates 15 stages. Tests use vi.hoisted() + vi.mock() for every imported agent/service/tool. Heavy mock setup (~30 mock variables) but fully tested.
services/transcription.ts5/10Calls Whisper client + FFmpeg audio extraction + fs writes. All mockable via module boundaries.
services/captionGeneration.ts6/10Thin wrapper — calls pure captionGenerator functions + writeTextFile. Easy to mock at fs level.
services/costTracker.ts5/10Singleton class exported as const costTracker = new CostTracker(). Tests call .reset() before each test — works, but shared mutable state requires discipline.
services/postStore.ts5/10All functions use getConfig() for queue dir + core/fileSystem for reads/writes. Tested via fs mocks.
services/processingState.ts5/10Same pattern — getConfig() + core/fileSystem. State file path derived at call time.
services/queueBuilder.ts5/10Depends on postStore, platformContentStrategy, core/fileSystem, types.
services/gitOperations.ts4/10Uses execCommandSync (shell exec). Must mock core/process.js.
services/lateApi.ts5/10HTTP client class. Constructor reads getConfig(). API calls go through fetch. Mock core/network.js or getConfig.
services/scheduler.ts5/10Composes lateApi + scheduleConfig + postStore. Three service mocks needed.
services/scheduleConfig.ts6/10Reads/writes schedule.json via core/fileSystem. Pure validation logic + fs I/O.
services/accountMapping.ts5/10Late API client + fs cache file. Needs module mocks for both.
services/fileWatcher.ts4/10Constructor calls getConfig() and fileExistsSync(). Extends EventEmitter. Chokidar dependency.
tools/ffmpeg/silenceDetection.ts5/10Uses createFFmpeg() from core. Mock core/ffmpeg.js.
tools/ffmpeg/captionBurning.ts5/10execFileRaw + temp dir + fs operations. Tests mock core/process.js + core/fileSystem.js.
tools/ffmpeg/singlePassEdit.ts5/10Pure buildFilterComplex + impure singlePassEditAndCaption (execFileRaw). Split already helps.
tools/ffmpeg/clipExtraction.ts5/10execFileRaw + ffprobe. Standard module mock.
tools/ffmpeg/aspectRatio.ts5/10execFileRaw + face detection integration.
tools/ffmpeg/audioExtraction.ts5/10createFFmpeg() fluent API.
tools/ffmpeg/frameCapture.ts5/10createFFmpeg() fluent API.
tools/ffmpeg/overlayCompositing.ts6/10Mix of pure getOverlayPosition (Tier 1) and impure compositeOverlays (execFileRaw).
tools/gemini/geminiClient.ts5/10Creates GoogleGenAI client from config. API calls + cost tracking. Mock @google/genai + getConfig.
tools/whisper/whisperClient.ts5/10Creates OpenAI client from config. File existence checks + API call. Mock core/ai.js + core/fileSystem.js.
tools/imageGeneration.ts5/10OpenAI DALL-E API + sharp for image processing + cost tracking.
tools/agentTools.ts5/10Utility functions wrapping ffprobe, fs reads, DALL-E.
stages/visualEnhancement.ts5/10Composes Gemini + GraphicsAgent + FFmpeg overlay. Three module mocks.
All Agent subclasses5/10Each extends BaseAgent (Tier 2) but tool handlers call Tier 3 tools/services. Mock via vi.mock() on dependencies.

Tier 4 — Hard to Mock (Score 1–3)

Singleton side effects, cached state, or import-time execution that makes testing difficult.

ModuleScoreNotes
config/environment.ts3/10Import side effect: lines 6–9 run loadEnvFile() at import time, mutating process.env. getConfig() returns a cached singleton (let config). Tests must call initConfig() to override, and env var stubs via vi.stubEnv() are fragile.
config/brand.ts3/10getBrandConfig() caches in let cachedBrand. Reads from fs on first call. No resetBrandConfig() exposed — tests must mock the entire module.
core/logger.ts3/10Winston logger created at import time (winston.createLogger()). Global singleton. Every module imports it. Tests mock it per-file with vi.mock('../config/logger.js').
core/ffmpeg.ts3/10getFFmpegPath() and getFFprobePath() call getConfig() + require('ffmpeg-static') + existsSync() at call time. Some tools call them at module top level (e.g. const ffmpegPath = getFFmpegPath() in captionBurning.ts, clipExtraction.ts, singlePassEdit.ts), making those modules hard to test without module mocks.
tools/ffmpeg/faceDetection.ts2/10`let cachedSession: ort.InferenceSession
providers/index.ts3/10getProvider() caches in let currentProvider (singleton). Has resetProvider() for testing (good), but factory calls getConfig().LLM_PROVIDER + logger.warn/info (side effects in factory).
config/modelConfig.ts3/10getModelForAgent() reads process.env directly (dynamic key MODEL_${name}) and falls through to getConfig().LLM_MODEL. Hard to isolate without env stubs.
src/index.ts (CLI entry)1/10Reads package.json synchronously at import. Constructs Commander program. Calls initConfig(), validateRequiredKeys(), starts watcher. Not unit-testable — integration/smoke test only.

3. Dependency Flow Analysis

CLI Entry (src/index.ts)

  • Depends on: Commander, config/environment, config/logger, services/fileWatcher, pipeline, services/processingState, core/fileSystem, core/paths
  • Consumed by: Node.js process entry point
  • Mock boundary: Not testable in isolation. Smoke tests validate CLI flags via subprocess.

Pipeline (src/pipeline.ts)

  • Depends on: All agents (Summary, Shorts, MediumVideo, Social, Blog, Chapter, Producer), all services (transcription, captionGeneration, costTracker, gitOperations, queueBuilder, processingState), tools (captionBurning, singlePassEdit), stages (visualEnhancement), config (environment, logger, modelConfig), core (fileSystem, paths)
  • Consumed by: src/index.ts, processVideoSafe()
  • Mock boundary: vi.mock() on every imported module. Tests use ~30 hoisted mock variables. runStage() and adjustTranscript() are independently testable.

Agents (src/agents/*)

  • Depends on: BaseAgentLLMProvider (constructor), providers/index (default), config/modelConfig, services/costTracker, config/logger
  • Consumed by: Pipeline stages, each other (SocialMediaAgent used for shorts + medium clips)
  • Mock boundary: LLMProvider interface — inject mock provider via constructor. Tool handlers are the impure seam (they call tools/services).

LLM Providers (src/providers/*)

  • Depends on: Respective SDK (@github/copilot-sdk, openai, @anthropic-ai/sdk), config/environment, config/logger
  • Consumed by: providers/index.ts factory, BaseAgent constructor
  • Mock boundary: Mock the SDK module, or inject a custom LLMProvider implementation.

Tools (src/tools/*)

  • Depends on: core/process (execFile), core/ffmpeg (path resolution), core/fileSystem, config/logger, services/costTracker (Gemini/Whisper), config/environment (API keys)
  • Consumed by: Agents (via tool handlers), services (transcription, captionGeneration), stages
  • Mock boundary: vi.mock('../../core/process.js') for FFmpeg tools. vi.mock('@google/genai') for Gemini. vi.mock('../../core/ai.js') for Whisper.

Services (src/services/*)

  • Depends on: config/environment, config/logger, core/fileSystem, core/paths, tools (whisper, ffmpeg), external APIs (Late.co)
  • Consumed by: Pipeline, agents, CLI commands, each other (scheduler → lateApi + postStore)
  • Mock boundary: vi.mock() on core/fileSystem.js and core/process.js covers most I/O. Service-to-service deps need individual mocks.

Config (src/config/*)

  • Depends on: core/fileSystem (brand.ts), core/env (environment.ts), process.env
  • Consumed by: Everything — every module in the project imports config
  • Mock boundary: vi.mock('../config/environment.js') is the most common mock across tests. initConfig() allows test overrides. vi.stubEnv() for env vars.

Core (src/core/*)

  • Depends on: Node.js builtins (fs, path, child_process), third-party libs (winston, fluent-ffmpeg, sharp, onnxruntime-node, chokidar, tmp)
  • Consumed by: Everything above
  • Mock boundary: Leaf-level mocks. Tests mock core/fileSystem.js, core/process.js, core/ffmpeg.js to control all I/O.

4. Key Mockability Boundaries

Boundary 1: LLMProvider Interface (Tier 2)

The cleanest seam in the architecture. BaseAgent accepts an optional LLMProvider via its constructor:

constructor(
  protected readonly agentName: string,
  protected readonly systemPrompt: string,
  provider?: LLMProvider,  // ← inject mock here
  model?: string,
)

Tests can inject a mock provider that returns canned LLMResponse objects without hitting any API. The LLMSession interface (sendAndWait, on, close) is small and easy to stub.

Current gap: Agent tests in agents.test.ts mock @github/copilot-sdk at module level instead of injecting a mock LLMProvider. This misses the DI seam that already exists.

Boundary 2: vi.mock() Module Boundary (Tier 3)

The dominant testing pattern. Vitest ESM mocks using vi.hoisted() + vi.mock():

const { mockExecFile } = vi.hoisted(() => ({ mockExecFile: vi.fn() }))
vi.mock('../core/process.js', () => ({ execFileRaw: mockExecFile }))

Used for: all FFmpeg tools, Whisper/Gemini clients, fs operations, config, logger, services.

What works well: The core/ layer provides thin wrappers around Node builtins, creating stable mock targets. Mocking core/fileSystem.js or core/process.js controls all file I/O and child processes in one place.

Boundary 3: Pure Function Extraction (Tier 1)

Several modules already extract pure logic from impure wrappers:

ModulePure exportImpure export
singlePassEdit.tsbuildFilterComplex()singlePassEditAndCaption()
overlayCompositing.tsgetOverlayPosition()compositeOverlays()
captionGenerator.tsAll exports (generateSRT, generateVTT, generateStyledASS, …)None
pricing.tsAll exportsNone
platformContentStrategy.tsAll exportsNone
pipeline.tsadjustTranscript()processVideo()

Pattern to replicate: Extract pure computation from I/O-heavy functions. The buildFilterComplex pattern is exemplary — pure string construction tested without mocking FFmpeg.

Boundary 4: File System Abstraction (src/core/fileSystem.ts)

All file operations go through core/fileSystem.ts, which wraps Node's fs module:

  • readJsonFile, readTextFile, writeJsonFile, writeTextFile
  • fileExists, ensureDirectory, copyFile, moveFile, removeFile
  • makeTempDir, withTempDir

This gives tests a single mock target for all file I/O. One vi.mock('../core/fileSystem.js') controls 20+ functions.

Boundary 5: Process Execution (src/core/process.ts)

All child process execution goes through core/process.ts:

  • execFileRaw — used by all FFmpeg tools, face detection
  • execCommandSync — used by git operations
  • execCommand — used by agent tools

Mocking this one module eliminates all subprocess side effects.


5. Consolidated Scorecard

ModuleTierScoreKey Blocker
captionGenerator.ts19/10None — pure functions, zero I/O
pricing.ts110/10None — pure math on const data
adjustTranscript()110/10None — pure timestamp remapping
buildFilterComplex()110/10None — pure string builder
getOverlayPosition()110/10None — pure expression builder
platformContentStrategy.ts19/10None — pure data lookup
BaseAgent27/10Tool handler side effects; getProvider() fallback in constructor
CopilotProvider27/10SDK dependency; vi.mock('@github/copilot-sdk')
OpenAIProvider27/10SDK dependency; vi.mock('openai')
ClaudeProvider27/10SDK dependency; vi.mock('@anthropic-ai/sdk')
socialPosting.ts28/10Already has interface + placeholder impl
processVideo()35/1030+ mock variables for full pipeline test
transcription.ts35/10Whisper client + FFmpeg + fs writes
captionGeneration.ts36/10Thin wrapper; pure core + fs write
costTracker.ts35/10Mutable singleton; .reset() helps but shared state
postStore.ts35/10fs-backed; getConfig() for paths
processingState.ts35/10fs-backed JSON state machine
queueBuilder.ts35/10Composes postStore + platformContentStrategy
gitOperations.ts34/10Shell execution via execCommandSync
lateApi.ts35/10HTTP client; fetch + config
scheduler.ts35/10Composes lateApi + scheduleConfig + postStore
scheduleConfig.ts36/10Pure validation + fs I/O
accountMapping.ts35/10Late API + fs cache
fileWatcher.ts34/10Constructor side effects (getConfig, fs check)
silenceDetection.ts35/10fluent-ffmpeg wrapper
captionBurning.ts35/10execFileRaw + temp dirs
singlePassEditAndCaption()35/10execFileRaw (pure helper already extracted)
clipExtraction.ts35/10execFileRaw + ffprobe
aspectRatio.ts35/10execFileRaw + face detection
audioExtraction.ts35/10fluent-ffmpeg wrapper
frameCapture.ts35/10fluent-ffmpeg wrapper
overlayCompositing.ts36/10Mixed: pure position helper + impure composite
geminiClient.ts35/10@google/genai SDK + costTracker
whisperClient.ts35/10OpenAI SDK + fs checks + costTracker
imageGeneration.ts35/10OpenAI DALL-E + sharp + costTracker
agentTools.ts35/10ffprobe + fs + DALL-E
visualEnhancement.ts35/10Composes Gemini + GraphicsAgent + FFmpeg
Agent subclasses35/10BaseAgent is DI-ready but tool handlers are Tier 3
environment.ts43/10Import-time side effect; cached singleton
brand.ts43/10Cached singleton; no reset function
core/logger.ts43/10Winston singleton created at import
core/ffmpeg.ts43/10getConfig() + require() at call time; consumers cache at module level
faceDetection.ts42/10ONNX cached session; model file required; sharp + ffmpeg
providers/index.ts43/10Cached singleton factory; resetProvider() helps
modelConfig.ts43/10Direct process.env reads + getConfig()
src/index.ts41/10Import-time sync fs read; Commander setup; not unit-testable

6. Cross-Cutting Concerns

Global Singletons

SingletonLocationReset mechanismRisk
getConfig()config/environment.tsinitConfig() overwritesImport-time .env loading mutates process.env before tests can intervene
loggercore/logger.tsNone (mock entire module)Every module imports it; 30+ vi.mock('../config/logger.js') across test files
costTrackerservices/costTracker.ts.reset() methodShared mutable state; setAgent()/setStage() are implicit context
cachedBrandconfig/brand.tsNone exposedFirst call caches forever; tests must mock the whole module
currentProviderproviders/index.tsresetProvider()Factory + cache; switching providers closes old one
cachedSessiontools/ffmpeg/faceDetection.tsNone exposedONNX session loaded once; heavy native dependency

File System Coupling

Nearly every layer reads or writes files:

  • Config: .env, brand.json, schedule.json
  • Services: processing-state.json, publish-queue/*/metadata.json, .vidpipe-cache.json
  • Tools: Temp directories for FFmpeg operations, audio chunks, frame captures
  • Pipeline: transcript.json, producer-plan.json, clip-direction.md, cost-report.md
  • Agents: Social posts, blog posts, README — all written to disk

Mitigation: All file ops go through core/fileSystem.ts, providing a single mock point.

External Process Dependency

FFmpeg is required by 9 tool modules. All use either:

  • fluent-ffmpeg (silenceDetection, audioExtraction, frameCapture) — mock core/ffmpeg.ts
  • execFileRaw (captionBurning, singlePassEdit, clipExtraction, aspectRatio, overlayCompositing, faceDetection) — mock core/process.ts

Top-level caching problem: Several modules resolve FFmpeg paths at the module top level:

const ffmpegPath = getFFmpegPath()  // executed at import time

This means getConfig() runs before test setup, potentially reading real env vars.

API Client Creation

ClientCreatedConfig sourceMockability
OpenAI (Whisper)Per-call in whisperClient.tsgetConfig().OPENAI_API_KEY✅ Good — mock core/ai.js
OpenAI (DALL-E)Per-call in imageGeneration.tsgetConfig().OPENAI_API_KEY✅ Good — mock getConfig
GoogleGenAIPer-call in geminiClient.tsgetConfig().GEMINI_API_KEY✅ Good — mock @google/genai
LateApiClientConstructor in lateApi.tsgetConfig()⚠️ Config read in constructor
LLM ProviderSingleton via getProvider()getConfig().LLM_PROVIDER⚠️ Cached singleton

Per-call creation is good for mockability. Singleton caching is the recurring problem.


7. Current State Summary

Tier Distribution (by module count)

TierCount% of modulesDescription
Tier 1 (Pure)612%No mocking needed
Tier 2 (DI)510%Interface-injectable
Tier 3 (Module-mock)3163%Requires vi.mock()
Tier 4 (Hard)816%Singletons, import side effects

What Already Works Well

  1. core/ abstraction layer — wrapping Node builtins (fs, child_process, path) in thin modules gives tests stable, low-churn mock targets. One vi.mock('../core/fileSystem.js') covers 20+ functions.

  2. LLMProvider interface — the provider abstraction is well-designed for DI. BaseAgent accepts a provider in its constructor. The LLMSession contract is small (3 methods).

  3. Pure function extractionbuildFilterComplex, captionGenerator, pricing, adjustTranscript, platformContentStrategy are all excellent examples of extracting testable logic from I/O-heavy modules.

  4. costTracker.reset() — the singleton has an explicit reset method, making test isolation straightforward.

  5. resetProvider() — the provider factory exposes a test-only reset function.

  6. vi.hoisted() pattern — consistently used across test files for ESM-compatible mock setup.

Biggest Gaps

  1. Config import side effectsenvironment.ts runs .env loading at import time. This poisons process.env before tests can set up isolation. Every test file that imports anything touching config inherits this side effect.

  2. No resetBrandConfig()brand.ts caches with no way to clear it between tests.

  3. Top-level FFmpeg path resolution — modules like captionBurning.ts, clipExtraction.ts, and singlePassEdit.ts run const ffmpegPath = getFFmpegPath() at module top level, coupling import to config + filesystem.

  4. faceDetection.ts ONNX session — global cached session with no reset. Requires ONNX model file on disk. Tests skip entirely via describe.skipIf().

  5. Agent tests mock SDK instead of using DIagents.test.ts mocks @github/copilot-sdk at module level. This tests the mock, not the agent logic. Should inject a mock LLMProvider via the constructor instead.

  6. Heavy pipeline test setuppipeline.test.ts requires ~30 hoisted mock variables. This is a symptom of the pipeline function doing too much — no intermediate abstractions between the orchestrator and individual stages.