Pulse Intelligence

July 2, 2026 · View on GitHub

Pulse Patrol is available to everyone on the Community plan with BYOK (your own AI provider). Pro adds hands-on Patrol modes, issue investigation, governed fixes, verified outcomes, and 90-day history, while hosted Cloud carries those capabilities for hosted environments. Learn more at https://pulserelay.pro or see PULSE_PRO.md.

Overview

Pulse Intelligence is built around a shared Pulse Intelligence Core: Canonical context, governed actions, safety gates, approval state, action audit, and verification shared by Pulse Assistant, Pulse MCP, and Pulse Patrol.

That core is deliberately surfaced with Patrol as the primary built-in operator and Assistant plus MCP as access paths over the same governed capabilities:

  1. Pulse Patrol: Patrol is the first-party operations surface: it checks infrastructure, investigates issues, follows the chosen Patrol mode before acting, verifies outcomes, and records what happened.
  2. Pulse Assistant: The contextual explanation, approval, and handoff surface for Patrol findings, governed actions, verification, and operator questions. Affordances: tools and interactive questions.
  3. Pulse MCP: The external-agent adapter that projects canonical Pulse Intelligence capabilities as MCP tools. Affordances: tools, resources, prompts, and capability metadata.

These surfaces are built on the same action-driven architecture: the configured LLM owns diagnosis, prioritization, fix reasoning, and action choice; Pulse supplies context, capabilities, safety gates, approval state, and audit trails. Verification is part of the governed action lifecycle rather than a separate model-owned feature.

Not Just Another Chatbot

Pulse Assistant is a protocol-driven, safety-gated LLM tool surface that:

  • Provides governed context — attaches explicit resource mentions and recent session facts without rewriting user intent
  • Caches session facts — extracts bounded tool facts to avoid redundant queries during the current conversation
  • Enforces workflow invariants — FSM prevents dangerous state transitions
  • Supports parallel tool execution — efficient batch operations with concurrency control
  • Grounds answers in real tool work — visible tool traces, read-after-write verification, and transcript hygiene prevent unsupported execution claims from being treated as facts
  • Returns structured tool errors — the model can recover from clear, machine-readable failures

📖 For a deep technical dive into the Assistant architecture, see architecture/pulse-assistant-deep-dive.md.

Not Just Another Alerting System

Pulse Patrol is a scheduled and event-triggered governed operator that:

  • Assembles evidence from metrics, storage, backups, discovery, alerts, and resource timelines
  • Provides statistical context such as baselines, trend summaries, capacity estimates, and event relationships
  • Lets the configured LLM reason over that evidence and decide whether to call tools or report findings
  • Routes governed actions through approval, entitlement, policy, verification, and audit boundaries
  • Preserves operator feedback as context for future model runs without converting it into Pulse-authored fixes

All while running entirely on your infrastructure with BYOK for complete privacy.

📖 For a deep technical dive into the Patrol runtime, see architecture/pulse-patrol-deep-dive.md.

See architecture/pulse-assistant.md for the original safety architecture documentation.

Assistant And MCP

Pulse Assistant and pulse-mcp are sibling surfaces over Pulse Intelligence, not competing implementations, and neither replaces the other. Assistant remains the in-app Pro surface for current resource/finding/run handoffs, approval cards, governed action status, and operator-friendly timelines. pulse-mcp owns the external-agent bridge: it fetches /api/agent/capabilities, projects those canonical API capabilities as MCP tools, and preserves the same stable error envelopes and approval/audit contracts. New operational capabilities should be added to the canonical API manifest first, then consumed by Assistant or MCP as appropriate; MCP-only actions and Assistant-only copies of the same business logic are drift.

Pulse Patrol

Patrol is a scheduled model workflow that builds a rich, system-wide snapshot and gives your configured LLM the tools it needs to produce actionable findings.

How Patrol Works

Scheduled/Event Trigger
        │
        ▼
buildSeedContext()  ── infrastructure evidence and policy context
        │
        ▼
LLM analysis (with tools) ← pulse_storage, pulse_metrics, pulse_alerts, etc.
        │
        ▼
patrol_report_finding() / patrol_resolve_finding() ── model-owned finding lifecycle calls
        │
        ├── DetectSignals() ── deterministic evidence extraction from tool outputs
        │       │
        │       ▼
        │   Evaluation pass ── focused LLM review of unmatched evidence
        │
        ▼
model-reported findings ── validated, deduplicated, stored
        │
        ▼ (if configured)
MaybeInvestigateFinding() ── model investigation + governed fix planning/execution

What Patrol Sees

Every patrol run passes the LLM comprehensive context about your environment:

Data CategoryWhat's Included
Proxmox NodesStatus, CPU%, memory%, uptime, 24h/7d trend analysis
VMs & ContainersFull metrics, backup status, OCI images, historical trends, anomaly evidence
Storage PoolsUsage %, capacity estimates, type (ZFS/LVM/Ceph), growth rates
Docker/PodmanContainer counts, health states, unhealthy container lists
KubernetesNodes, pods, deployments, services, DaemonSets, StatefulSets, namespaces
TrueNASPools, datasets, disk health, SMART status, replication, alerts
PBS/PMGDatastore status, backup jobs, job failures, verification status
CephCluster health, OSD states, PG status
Agent HostsLoad averages, memory, disk, RAID status, temperatures

Model-Bound Context

Beyond raw metrics, Patrol prepares structured evidence for the model:

  • Trend summaries — 24h and 7d samples showing growing, stable, declining, or volatile behavior
  • Baseline evidence — Z-score anomaly evidence from historical metrics
  • Capacity estimates — "Storage pool reaches 95% in about 12 days at current growth rate"
  • Infrastructure changes — Detected config changes, VM migrations, new deployments
  • Resource relationships — Related events and topology context
  • User notes — Your annotations explaining expected behavior
  • Dismissed findings — Respects your feedback and suppressed alerts
  • Investigation context — Uses prior alert context, Patrol run history, and resource timelines

Deterministic Evidence Extraction

Patrol parses tool outputs for concrete evidence such as backup failures, storage pressure, and disk health failures. These signals are not final findings by themselves: unmatched signals are sent to a focused LLM evaluation pass, and if the model still declines to report them, Pulse does not convert them into Pulse-authored findings.

Signal TypeTriggerDefault Threshold
smart_failureSMART health status not OK/PASSED, or critical SMART counters such as pending sectors, offline uncorrectable sectors, or NVMe media errorsN/A
high_cpuAverage CPU usage70%
high_memoryAverage memory usage80%
high_diskStorage pool usage75% (warning), 95% (critical)
backup_failedRecent backup task with error statusWithin 48h
backup_staleNo backup completed for VM/CT48+ hours

Thresholds can be configured via alert settings to match user-defined values.

Examples of What Patrol Catches

IssueSeverityExample
Disk approaching capacityWarning/CriticalStorage growing toward full with concrete time-to-threshold evidence
Backup failuresWarningPBS job failed, no backup in 48+ hours
Storage issuesCriticalPBS datastore errors, ZFS pool degraded
Ceph problemsWarning/CriticalDegraded OSDs, unhealthy PGs
Kubernetes issuesWarningPods stuck in Pending/CrashLoopBackOff
SMART failuresCriticalDisk health check failed, pending sectors, offline uncorrectable sectors, or NVMe media errors
Alert-triggered investigationsPro / CloudA fired alert prompts the model to gather surrounding context and explain likely cause

What Patrol Ignores (by design)

Patrol is intentionally conservative to avoid noise:

  • Small baseline deviations ("CPU at 15% vs typical 10%")
  • Low utilization that's "elevated" but fine (disk at 40%)
  • Stopped VMs/containers that were intentionally stopped
  • Brief spikes that resolve on their own
  • Anything that doesn't require human action
  • Conditions already fully covered by the normal alert lifecycle unless the model finds additional context that changes the operator decision

Philosophy: If a finding wouldn't be worth waking someone up at 3am, Patrol won't create it.

Finding Severity

  • Critical: Immediate attention required (service down, data at risk)
  • Warning: Should be addressed soon (disk filling, backup stale)

Note: info and watch level findings are filtered out to reduce noise.

Managing Findings

Findings can be managed via the UI or API:

  • Get help: Chat with AI to troubleshoot the issue
  • Resolve: Mark as fixed (finding will reappear if the issue resurfaces)
  • Dismiss: Mark as expected behavior (creates suppression rule)

Dismissed and resolved findings persist across Pulse restarts.

Patrol Modes

Patrol supports four modes that decide how far Pulse can go after it finds an issue:

ModeBehaviorPlan
Watch onlyDetect issues only. No investigation or fixes.Community (BYOK)
Ask before changesInvestigates findings and proposes fixes. All fixes require approval before execution.Pro / hosted Cloud
Auto-fix safe issuesRuns warning-level governed fixes automatically and verifies results. Critical findings still require approval by default.Pro / hosted Cloud
Policy autopilotRuns eligible governed fixes automatically and verifies results. Use only in environments where this is acceptable.Pro / hosted Cloud

Community and Relay installs can still run scheduled Patrol findings with BYOK. Watch only remains the free-first baseline; investigation, proposed fixes, and fix execution are paid AI-operations capabilities rather than a core monitoring limit.

Investigation Flow

When a finding is created in a Pro Patrol mode:

Finding created
      │
      ▼
MaybeInvestigateFinding()
      │
      ├─ Has orch + chatService?
      │        │
      │        ▼
      │   InvestigateFinding()
      │        │
      │        ▼
      │   Create chat session
      │        │
      │        ▼
      │   AI analysis (with tools)
      │        │
      │        ▼
      │   [Fix proposed?] ──Yes──► Queue approval (or auto-execute in full mode)
      │        │
      │        No
      │        ▼
      │   Update finding with outcome
      │
      └─ Skip investigation

Investigation Configuration

SettingDefaultDescription
MaxTurns15Maximum agentic turns per investigation
Timeout10 minMaximum duration per investigation
MaxConcurrent3Maximum concurrent investigations
MaxAttemptsPerFinding3Maximum investigation attempts per finding
CooldownDuration1 hourCooldown before re-investigating
TimeoutCooldownDuration10 minShorter cooldown for timeout failures
VerificationDelay30 secWait before verifying fix

Investigation Outcomes

OutcomeMeaning
resolvedIssue resolved during investigation
fix_queuedFix proposed, awaiting approval
fix_executedFix auto-executed successfully
fix_failedFix attempted but failed
fix_verifiedFix worked, issue confirmed resolved
fix_verification_failedFix ran but issue persists
needs_attentionRequires human intervention
cannot_fixIssue cannot be automatically fixed
timed_outInvestigation timed out (will retry sooner)

Pulse Assistant (Chat)

Pulse Assistant is a tool-driven chat interface. It does not "guess" system state — it calls live tools and reports their outputs.

The Model's Workflow (Discover → Investigate → Act)

  1. Discover: Uses pulse_query or pulse_discovery to find real resources and IDs
  2. Investigate: Uses pulse_read to run bounded, read-only commands and check status/logs
  3. Act (optional): Uses pulse_control for changes, then verifies with a read

Tool Inventory

The Assistant tool list is registry-owned at runtime, not hand-maintained in this public overview. Each turn receives an available-tool manifest generated from Pulse's governed tool registry, including action mode (read, mixed, write) and approval policy (scope_only, action_plan). That same registry feeds the Assistant system prompt, provider tool declarations, tool-result handling, approval boundaries, and Patrol-only tool filtering.

For the current source-owned inventory, see the native tool registry and governance projection in internal/ai/tools/ and internal/agentcapabilities/. For external agents, use the live /api/agent/capabilities manifest or pulse-mcp tools/list; those surfaces project the canonical agent capabilities rather than a separate MCP-only tool table. The same manifest also carries reusable workflowPrompts metadata so Pulse Assistant-compatible starters and MCP prompts/list clients discover the same fleet triage, resource investigation, and Patrol finding review workflows.

Safety Gates

The assistant enforces multiple safety gates:

  1. Discovery Before Action — Action tools cannot operate on resources that weren't first discovered
  2. Verification After Write — After any write, the model must perform a read/status check before providing a final answer
  3. Read/Write Separation — Read operations route through pulse_read (stays in READING state); write operations route through pulse_control (enters VERIFYING state)
  4. Grounded Execution Guardrails — Visible tool traces and read-after-write checks prevent unsupported execution claims from being treated as facts
  5. Approval Mode — In Controlled mode, every write requires explicit user approval
  6. Execution Context Binding — Commands execute within the resolved resource's context, not on parent hosts

Control Levels

LevelBehaviorPlan
Read-onlyAI can observe and query data onlyCommunity
ControlledAI asks for approval before executing commandsCommunity
AutonomousAI executes actions without promptingPro / hosted Cloud

Using Approvals (Controlled Mode)

When control level is Controlled, write actions pause for approval:

  1. Tool returns APPROVAL_REQUIRED: { approval_id, command, ... }
  2. Agentic loop emits approval_needed SSE event
  3. UI shows approval card with the proposed command
  4. Approve to execute and verify, or Deny to cancel
  5. Only users with admin privileges can approve/deny

Configuration

Configure providers in the UI: Settings → Pulse Intelligence → Provider & Models

Supported Providers

  • Anthropic (API key)
  • OpenAI
  • OpenRouter
  • DeepSeek
  • Google Gemini
  • Ollama (self-hosted, with tool/function calling support)
  • OpenAI-compatible base URL (for providers that implement the OpenAI API shape)

Legacy Anthropic OAuth fields may still appear in stored settings so existing installs can disconnect and clear old tokens, but Anthropic OAuth is not a supported runtime authentication method and does not make Anthropic configured.

Models

Pulse uses model identifiers in the form: provider:model-name

You can set separate models for:

  • Chat (chat_model)
  • Patrol (patrol_model)
  • Patrol fix model (auto_fix_model, retained as the compatibility settings key)

Storage

AI settings are stored encrypted at rest in ai.enc under the Pulse config directory. Related files:

FilePurpose
ai.encEncrypted AI configuration and credentials
ai_findings.jsonPatrol findings
ai_patrol_runs.jsonPatrol run history
ai_usage_history.jsonToken usage data
ai_chat_sessions.jsonLegacy chat sessions (UI sync)
baselines.jsonLearned resource baselines
ai_correlations.jsonResource correlation data
ai_patterns.jsonDetected patterns

Config directory: /etc/pulse (systemd) or /data (Docker/Kubernetes)

Testing

  • Test provider connectivity: POST /api/ai/test and POST /api/ai/test/{provider}
  • List available models: GET /api/ai/models

Schedule and Triggers

Patrol runs on a configurable schedule:

IntervalDescription
DisabledPatrol runs only when manually triggered
10 min – 7 daysConfigurable interval (default: 6 hours)

Patrol can also be triggered by:

  • Manual run: Click "Run Patrol" in the UI
  • Alert-triggered analysis (Pro and above): Runs when an alert fires
  • API call: POST /api/ai/patrol/run

Model Context Layer

Pulse includes a model-context layer that aggregates evidence from AI runtime subsystems:

Components

ComponentPurpose
Baseline StoreMaintains statistical metric summaries and anomaly evidence
Pattern StoreRecords recurring event evidence and trend context
Correlation StoreLinks related events and resource relationships for model context
Investigation ContextUses alert history, Patrol runs, and resource timelines
Knowledge StorePersists user annotations and model-safe context
Forecast ServiceEstimates capacity trajectories from historical samples

Health Scoring

The Patrol UI can show an operational score (A-F) based on active findings, Patrol coverage, runtime errors, and structured evidence. This score is a presentation aid, not a replacement for model diagnosis.

Model Matrix (Pulse Assistant)

This table summarizes the most recent Pulse Assistant eval runs per model.

Update the table from eval reports:

EVAL_REPORT_DIR=tmp/eval-reports go run ./cmd/eval -scenario matrix -auto-models
python3 scripts/eval/render_model_matrix.py tmp/eval-reports --write-doc docs/AI.md

Or use the helper script:

scripts/eval/run_model_matrix.sh

Run the resource-context Assistant handoff eval against a live resource:

EVAL_RESOURCE_CONTEXT_ID=delly:delly:101 \
EVAL_RESOURCE_CONTEXT_NAME=homeassistant \
EVAL_RESOURCE_CONTEXT_TYPE=system-container \
EVAL_RESOURCE_CONTEXT_NODE=delly \
EVAL_RESOURCE_CONTEXT_FORBIDDEN="/mnt/pve/finance-db,/var/lib/homeassistant,literal-provider-token-123" \
go run ./cmd/eval -scenario resource-context -url http://127.0.0.1:7655 -user admin -pass "$PULSE_EVAL_PASS"
ModelSmokeRead-onlyTime (matrix)Tokens (matrix)Last run (UTC)
anthropic:claude-3-haiku-20240307✅❌2m 42s—2026-01-29
anthropic:claude-haiku-4-5-20251001✅✅8s18,9232026-01-29
anthropic:claude-opus-4-5-20251101✅✅9m 31s1,120,5302026-01-29
gemini:gemini-3-flash-preview✅✅7m 4s—2026-01-29
gemini:gemini-3-pro-preview✅✅3m 54s1,9142026-01-29
openai:gpt-5.2✅✅5s12,3632026-01-29
openai:gpt-5.2-chat-latest✅✅8s12,5952026-01-29

Safety Controls

Pulse includes settings that control how "active" AI features are:

  • Patrol modes (Pro and above): Lets you choose whether Patrol only watches, asks before changes, handles safe fixes, or uses policy autopilot
  • Governed fixes (Pro and above): Allows Patrol to propose, approve, run, verify, and record fixes under the Patrol mode you choose
  • Issue investigation (Pro and above): Lets Patrol investigate findings with surrounding infrastructure context
  • Policy autopilot unlock (Pro and above): Permits eligible critical fixes without per-fix approval after an explicit opt-in

If you enable execution features, ensure agent tokens and scopes are appropriately restricted.

Advanced Network Restrictions

Pulse blocks AI tool HTTP fetches to loopback and link-local addresses by default. For local development:

  • PULSE_AI_ALLOW_LOOPBACK=true

Use this only in trusted environments.

Privacy

Patrol runs on your server and only sends the minimal context needed for analysis to the configured provider (when AI is enabled). Outbound usage telemetry (a rotating pseudonymous install ID, counts, feature flags, and coarse Patrol mode and governed Pulse Intelligence operations adoption flags and counters only; no hostnames, credentials, prompts, chat messages, command text, action output, token values, IP addresses, or resource identifiers in the payload) is enabled by default and can be disabled any time. See Privacy for details.

Why Patrol Is Different From Traditional Alerts

Alerts are threshold-based and narrow. Patrol gives the selected model a broader, tool-backed operating picture.

  • Alerts: "Disk > 90%"
  • Patrol: "The model sees ZFS pool usage, growth rate, datastore consumers, backup context, and governed actions, then decides whether that evidence warrants a finding or action recommendation."

Cost Tracking

Pulse tracks token usage and costs:

  • View usage summary: GET /api/ai/cost/summary
  • Reset counters: POST /api/ai/cost/reset (admin)
  • Set monthly budget limits in AI settings

Troubleshooting

IssueSolution
Assistant or Patrol not respondingVerify provider credentials in Settings → Pulse Intelligence → Provider & Models
No execution capabilityConfirm at least one agent is connected
Findings not persistingCheck Pulse has write access to ai_findings.json in the config directory
Too many findingsThis shouldn't happen — please report if it does
Investigation stuckCheck circuit breaker status at /api/ai/circuit/status; may auto-reset after cooldown
Model not availableEnsure provider API key is valid and model ID matches provider format
  • Pulse Assistant Deep Dive — Complete technical breakdown of the model-owned tool surface: explicit context, session fact caching, FSM enforcement, parallel execution, grounded execution guardrails, structured errors
  • Pulse Patrol Deep Dive — Patrol runtime documentation: evidence assembly, deterministic signal extraction, model evaluation, investigation context, investigation orchestration

Reference Documentation