Database Documentation

June 12, 2026 ยท View on GitHub

Comprehensive reference for OwnPilot's PostgreSQL database layer. This document covers the schema, all tables, the repository pattern, data stores, seed data, migrations, connection management, and relationship model.


Table of Contents

  1. Overview
  2. Schema Definition
  3. Table Reference
  4. Relationships and Entity Diagram
  5. Index Strategy
  6. Repository Pattern
  7. Data Stores
  8. Seed Data
  9. Migrations
  10. Connection Management
  11. Docker Compose Setup
  12. Environment Variables

1. Overview

OwnPilot uses PostgreSQL 16 as its sole production database. The database layer lives inside the packages/gateway package and follows a strict layered architecture:

packages/gateway/src/db/
  adapters/              # Database adapter abstraction
    types.ts             # DatabaseAdapter interface, DatabaseConfig, getDatabaseConfig()
    postgres-adapter.ts  # PostgresAdapter implementation using pg Pool
    index.ts             # Adapter singleton (getAdapter, initializeAdapter, closeAdapter)
  repositories/          # One repository per domain (extends BaseRepository)
    base.ts              # BaseRepository abstract class
    index.ts             # Re-exports all repositories
    conversations.ts
    messages.ts
    agents.ts
    ... (40+ repository files)
  seeds/                 # Seed data modules
    default-agents.ts    # Loads agent configs from JSON
    plans-seed.ts        # Example plan seeds
    config-services-seed.ts  # Known service definitions
    index.ts             # seedDefaultAgents(), runSeeds()
  schema.ts              # SCHEMA_SQL, MIGRATIONS_SQL, INDEXES_SQL, initializeSchema()
  data-stores.ts         # DataStore<T> implementations wrapping repositories

Key Design Decisions

  • Text primary keys -- Every table uses TEXT PRIMARY KEY with UUIDs or human-readable identifiers, never auto-incrementing integers.
  • JSONB for flexible data -- Columns like metadata, config, tags, tool_calls, and parameters use PostgreSQL JSONB for schema-flexible storage.
  • Idempotent schema creation -- All CREATE TABLE and CREATE INDEX statements use IF NOT EXISTS. Migrations use DO $$ ... END $$ blocks that check information_schema.columns before adding columns.
  • Cascade deletes for child tables -- Child records (messages, steps, logs) are automatically deleted when their parent is removed.
  • Timestamps via NOW() -- All tables use TIMESTAMP NOT NULL DEFAULT NOW() for created_at; most also track updated_at.

2. Schema Definition

Source file: packages/gateway/src/db/schema.ts

The schema is defined as three exported SQL template literals and one initialization function:

2.1 SCHEMA_SQL

Contains all CREATE TABLE IF NOT EXISTS statements. Tables are grouped by domain with SQL comments marking each section. This string creates every table from scratch when executed against an empty database.

2.2 MIGRATIONS_SQL

Contains idempotent ALTER TABLE statements wrapped in PL/pgSQL DO $$ ... END $$ blocks. Each block checks information_schema.columns before adding a column, making it safe to run repeatedly. Also includes:

  • Table creation for config_services and config_entries (created in migrations because they replaced the older api_services table).
  • Data migration logic from the deprecated api_services table to config_services / config_entries, including automatic DROP TABLE api_services after migration.
  • Creation of plugins, local_providers, and local_models tables.

2.3 INDEXES_SQL

Contains all CREATE INDEX IF NOT EXISTS statements. Over 100 indexes are defined, organized by table group. See Section 5 for the full strategy.

2.4 initializeSchema()

export async function initializeSchema(exec: (sql: string) => Promise<void>): Promise<void>;

Runs all three SQL blocks in order:

  1. SCHEMA_SQL -- creates tables
  2. MIGRATIONS_SQL -- applies column additions and data migrations
  3. INDEXES_SQL -- creates indexes

This function is called automatically on the first database connection via the adapter initialization path (adapters/index.ts -> createAdapter()).


3. Table Reference

OwnPilot defines 47 tables organized into 11 domain groups. Every table uses TEXT PRIMARY KEY.


3.1 Core Tables

These tables support the primary chat and agent functionality.

conversations

Stores chat conversation sessions.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYConversation UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
titleTEXTConversation title
agent_idTEXTAssociated agent
agent_nameTEXTAgent display name
providerTEXTAI provider used
modelTEXTAI model used
system_promptTEXTCustom system prompt
message_countINTEGERNOT NULL DEFAULT 0Cached message count
is_archivedBOOLEANNOT NULL DEFAULT FALSEArchive flag
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update time
metadataJSONBDEFAULT '{}'Arbitrary metadata

messages

Individual messages within a conversation.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYMessage UUID
conversation_idTEXTNOT NULL, FK -> conversations(id) ON DELETE CASCADEParent conversation
roleTEXTNOT NULL, CHECK IN ('system','user','assistant','tool')Message role
contentTEXTNOT NULLMessage body
providerTEXTAI provider
modelTEXTAI model
tool_callsJSONBTool call definitions
tool_call_idTEXTTool call response ID
traceTEXTDebug trace
is_errorBOOLEANNOT NULL DEFAULT FALSEError indicator
input_tokensINTEGERToken usage (input)
output_tokensINTEGERToken usage (output)
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time

request_logs

HTTP request and API call logging for debugging.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYLog entry UUID
user_idTEXTNOT NULL DEFAULT 'default'User making request
conversation_idTEXTFK -> conversations(id) ON DELETE SET NULLRelated conversation
typeTEXTNOT NULL, CHECK IN ('chat','completion','embedding','tool','agent','other')Request type
providerTEXTAI provider
modelTEXTAI model
endpointTEXTTarget endpoint
methodTEXTNOT NULL DEFAULT 'POST'HTTP method
request_bodyJSONBRequest payload
response_bodyJSONBResponse payload
status_codeINTEGERHTTP status code
input_tokensINTEGERToken usage (input)
output_tokensINTEGERToken usage (output)
total_tokensINTEGERToken usage (total)
duration_msINTEGERRequest duration
errorTEXTError message
error_stackTEXTError stack trace
ip_addressTEXTClient IP
user_agentTEXTClient user agent
created_atTIMESTAMPNOT NULL DEFAULT NOW()Log time

channels

External communication channel configurations (Telegram).

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYChannel UUID
typeTEXTNOT NULLChannel type
nameTEXTNOT NULLDisplay name
statusTEXTNOT NULL DEFAULT 'disconnected'Connection status
configJSONBNOT NULL DEFAULT '{}'Channel configuration
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
connected_atTIMESTAMPLast connection time
last_activity_atTIMESTAMPLast activity time

channel_messages

Messages received from or sent to external channels.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYMessage UUID
channel_idTEXTNOT NULL, FK -> channels(id) ON DELETE CASCADEParent channel
external_idTEXTPlatform message ID
directionTEXTNOT NULL, CHECK IN ('inbound','outbound')Message direction
sender_idTEXTSender identifier
sender_nameTEXTSender name
contentTEXTNOT NULLMessage body
content_typeTEXTNOT NULL DEFAULT 'text'Content type
attachmentsJSONBFile attachments
reply_to_idTEXTReply target
metadataJSONBDEFAULT '{}'Extra metadata
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time

costs

AI API usage cost tracking.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYCost entry UUID
providerTEXTNOT NULLAI provider
modelTEXTNOT NULLAI model
conversation_idTEXTFK -> conversations(id) ON DELETE SET NULLRelated conversation
input_tokensINTEGERNOT NULL DEFAULT 0Input tokens consumed
output_tokensINTEGERNOT NULL DEFAULT 0Output tokens consumed
total_tokensINTEGERNOT NULL DEFAULT 0Total tokens
input_costREALNOT NULL DEFAULT 0Input cost in dollars
output_costREALNOT NULL DEFAULT 0Output cost in dollars
total_costREALNOT NULL DEFAULT 0Total cost
created_atTIMESTAMPNOT NULL DEFAULT NOW()Time of API call

agents

Pre-configured AI agent profiles with system prompts and tool access.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYAgent identifier
nameTEXTNOT NULL UNIQUEDisplay name
system_promptTEXTSystem prompt template
providerTEXTNOT NULLDefault AI provider
modelTEXTNOT NULLDefault AI model
configJSONBNOT NULL DEFAULT '{}'Agent configuration (maxTokens, temperature, tools, toolGroups)
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update time

settings

Global key-value settings store.

ColumnTypeConstraintsDescription
keyTEXTPRIMARY KEYSetting key
valueTEXTNOT NULLSetting value
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update time

3.2 Personal Data Tables

User-facing data management for personal productivity.

bookmarks

Web bookmark storage with categorization and visit tracking.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYBookmark UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
urlTEXTNOT NULLBookmark URL
titleTEXTNOT NULLPage title
descriptionTEXTDescription
faviconTEXTFavicon URL
categoryTEXTCategory
tagsJSONBDEFAULT '[]'Tag array
is_favoriteBOOLEANNOT NULL DEFAULT FALSEFavorite flag
visit_countINTEGERNOT NULL DEFAULT 0Visit counter
last_visited_atTIMESTAMPLast visit
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

notes

Rich-text notes with markdown support.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYNote UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
titleTEXTNOT NULLNote title
contentTEXTNOT NULLNote body
content_typeTEXTNOT NULL DEFAULT 'markdown'Content format
categoryTEXTCategory
tagsJSONBDEFAULT '[]'Tag array
is_pinnedBOOLEANNOT NULL DEFAULT FALSEPin flag
is_archivedBOOLEANNOT NULL DEFAULT FALSEArchive flag
colorTEXTDisplay color
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

tasks

Task management with subtask hierarchy and project grouping.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYTask UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
titleTEXTNOT NULLTask title
descriptionTEXTTask details
statusTEXTNOT NULL DEFAULT 'pending', CHECK IN ('pending','in_progress','completed','cancelled')Current status
priorityTEXTNOT NULL DEFAULT 'normal', CHECK IN ('low','normal','high','urgent')Priority level
due_dateTIMESTAMPDue date
due_timeTEXTDue time string
reminder_atTIMESTAMPReminder time
categoryTEXTCategory
tagsJSONBDEFAULT '[]'Tag array
parent_idTEXTFK -> tasks(id) ON DELETE SET NULLParent task (self-referencing)
project_idTEXTAssociated project
recurrenceTEXTRecurrence rule
completed_atTIMESTAMPCompletion time
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

calendar_events

Personal calendar event storage with recurrence and attendee support.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYEvent UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
titleTEXTNOT NULLEvent title
descriptionTEXTEvent details
locationTEXTLocation
start_timeTIMESTAMPNOT NULLStart time
end_timeTIMESTAMPEnd time
all_dayBOOLEANNOT NULL DEFAULT FALSEAll-day flag
timezoneTEXTDEFAULT 'UTC'Timezone
recurrenceTEXTRecurrence rule
reminder_minutesINTEGERMinutes before to remind
categoryTEXTCategory
tagsJSONBDEFAULT '[]'Tag array
colorTEXTDisplay color
external_idTEXTExternal service ID
external_sourceTEXTExternal service name
attendeesJSONBDEFAULT '[]'Attendee list
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

contacts

Contact information with social links and custom fields.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYContact UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
nameTEXTNOT NULLFull name
nicknameTEXTNickname
emailTEXTEmail address
phoneTEXTPhone number
companyTEXTCompany name
job_titleTEXTJob title
avatarTEXTAvatar URL
birthdayTEXTBirthday string
addressTEXTPhysical address
notesTEXTContact notes
relationshipTEXTRelationship type
tagsJSONBDEFAULT '[]'Tag array
is_favoriteBOOLEANNOT NULL DEFAULT FALSEFavorite flag
external_idTEXTExternal service ID
external_sourceTEXTExternal service name
social_linksJSONBDEFAULT '{}'Social media links
custom_fieldsJSONBDEFAULT '{}'User-defined fields
last_contacted_atTIMESTAMPLast contact time
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

projects

Project containers for grouping tasks.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYProject UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
nameTEXTNOT NULLProject name
descriptionTEXTProject details
colorTEXTDisplay color
iconTEXTDisplay icon
statusTEXTNOT NULL DEFAULT 'active', CHECK IN ('active','completed','archived')Project status
due_dateTIMESTAMPDue date
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

reminders

Standalone reminders with optional entity linking.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYReminder UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
titleTEXTNOT NULLReminder title
descriptionTEXTReminder details
remind_atTIMESTAMPNOT NULLTarget time
recurrenceTEXTRecurrence rule
is_completedBOOLEANNOT NULL DEFAULT FALSECompletion flag
related_typeTEXTLinked entity type
related_idTEXTLinked entity ID
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

captures

Quick-capture inbox for ideas, thoughts, and snippets.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYCapture UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
contentTEXTNOT NULLCapture text
typeTEXTNOT NULL DEFAULT 'thought', CHECK IN ('idea','thought','todo','link','quote','snippet','question','other')Capture type
tagsJSONBDEFAULT '[]'Tag array
sourceTEXTOrigin source
urlTEXTRelated URL
processedBOOLEANNOT NULL DEFAULT FALSEProcessing flag
processed_as_typeTEXTCHECK IN ('note','task','bookmark','discarded') OR NULLWhat it became
processed_as_idTEXTID of created entity
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
processed_atTIMESTAMPProcessing time

3.3 Productivity Tables

Time tracking, focus sessions, and habit building.

pomodoro_sessions

Individual Pomodoro timer sessions.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYSession UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
typeTEXTNOT NULL, CHECK IN ('work','short_break','long_break')Session type
statusTEXTNOT NULL DEFAULT 'running', CHECK IN ('running','completed','interrupted')Current status
task_descriptionTEXTWhat the user is working on
duration_minutesINTEGERNOT NULLPlanned duration
started_atTIMESTAMPNOT NULL DEFAULT NOW()Start time
completed_atTIMESTAMPCompletion time
interrupted_atTIMESTAMPInterruption time
interruption_reasonTEXTWhy it was interrupted

pomodoro_settings

Per-user Pomodoro configuration.

ColumnTypeConstraintsDescription
user_idTEXTPRIMARY KEY DEFAULT 'default'User ID (primary key)
work_durationINTEGERNOT NULL DEFAULT 25Work session minutes
short_break_durationINTEGERNOT NULL DEFAULT 5Short break minutes
long_break_durationINTEGERNOT NULL DEFAULT 15Long break minutes
sessions_before_long_breakINTEGERNOT NULL DEFAULT 4Sessions before long break
auto_start_breaksBOOLEANNOT NULL DEFAULT FALSEAuto-start breaks
auto_start_workBOOLEANNOT NULL DEFAULT FALSEAuto-start work
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

pomodoro_daily_stats

Aggregated daily Pomodoro statistics for streak tracking.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYStats entry UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
dateTEXTNOT NULLDate string (YYYY-MM-DD)
completed_sessionsINTEGERNOT NULL DEFAULT 0Sessions completed
total_work_minutesINTEGERNOT NULL DEFAULT 0Work time
total_break_minutesINTEGERNOT NULL DEFAULT 0Break time
interruptionsINTEGERNOT NULL DEFAULT 0Interruption count

UNIQUE constraint on (user_id, date).

habits

Habit definitions with streak tracking.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYHabit UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
nameTEXTNOT NULLHabit name
descriptionTEXTHabit details
frequencyTEXTNOT NULL DEFAULT 'daily', CHECK IN ('daily','weekly','weekdays','custom')Recurrence pattern
target_daysJSONBDEFAULT '[]'Days of week for custom
target_countINTEGERNOT NULL DEFAULT 1Target per period
unitTEXTMeasurement unit
categoryTEXTCategory
colorTEXTDisplay color
iconTEXTDisplay icon
reminder_timeTEXTDaily reminder time
is_archivedBOOLEANNOT NULL DEFAULT FALSEArchive flag
streak_currentINTEGERNOT NULL DEFAULT 0Current streak
streak_longestINTEGERNOT NULL DEFAULT 0Best streak
total_completionsINTEGERNOT NULL DEFAULT 0Lifetime completions
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

habit_logs

Daily habit completion records.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYLog entry UUID
habit_idTEXTNOT NULL, FK -> habits(id) ON DELETE CASCADEParent habit
user_idTEXTNOT NULL DEFAULT 'default'Owner user
dateTEXTNOT NULLDate string (YYYY-MM-DD)
countINTEGERNOT NULL DEFAULT 1Completion count
notesTEXTOptional notes
logged_atTIMESTAMPNOT NULL DEFAULT NOW()Log time

UNIQUE constraint on (habit_id, date).


3.4 Autonomous AI Tables

Support for persistent AI memory, goal tracking, proactive triggers, and autonomous plan execution.

memories

Persistent AI memory store for facts, preferences, and context.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYMemory UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
typeTEXTNOT NULL, CHECK IN ('fact','preference','conversation','event','skill')Memory type
contentTEXTNOT NULLMemory content
embeddingBYTEAVector embedding (binary)
sourceTEXTWhere memory originated
source_idTEXTSource entity ID
importanceREALNOT NULL DEFAULT 0.5, CHECK >= 0 AND <= 1Importance score (0-1)
tagsJSONBDEFAULT '[]'Tag array
accessed_countINTEGERNOT NULL DEFAULT 0Access counter
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update
accessed_atTIMESTAMPLast access time
metadataJSONBDEFAULT '{}'Extra metadata

goals

Long-term objectives with hierarchical sub-goal support.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYGoal UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
titleTEXTNOT NULLGoal title
descriptionTEXTGoal details
statusTEXTNOT NULL DEFAULT 'active', CHECK IN ('active','paused','completed','abandoned')Goal status
priorityINTEGERNOT NULL DEFAULT 5, CHECK >= 1 AND <= 10Priority (1-10)
parent_idTEXTFK -> goals(id) ON DELETE SET NULLParent goal (self-referencing)
due_dateTIMESTAMPDue date
progressREALNOT NULL DEFAULT 0, CHECK >= 0 AND <= 100Completion percent
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update
completed_atTIMESTAMPCompletion time
metadataJSONBDEFAULT '{}'Extra metadata

goal_steps

Actionable steps toward completing a goal.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYStep UUID
goal_idTEXTNOT NULL, FK -> goals(id) ON DELETE CASCADEParent goal
titleTEXTNOT NULLStep title
descriptionTEXTStep details
statusTEXTNOT NULL DEFAULT 'pending', CHECK IN ('pending','in_progress','completed','blocked','skipped')Step status
order_numINTEGERNOT NULLExecution order
dependenciesJSONBDEFAULT '[]'Dependent step IDs
resultTEXTExecution result
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
completed_atTIMESTAMPCompletion time

triggers

Proactive automation triggers (scheduled, event-based, conditional, webhook).

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYTrigger UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
nameTEXTNOT NULLTrigger name
descriptionTEXTTrigger details
typeTEXTNOT NULL, CHECK IN ('schedule','event','condition','webhook')Trigger type
configJSONBNOT NULL DEFAULT '{}'Type-specific config (cron, eventType, condition, secret)
actionJSONBNOT NULL DEFAULT '{}'Action to execute (type + payload)
enabledBOOLEANNOT NULL DEFAULT TRUEActive flag
priorityINTEGERNOT NULL DEFAULT 5, CHECK >= 1 AND <= 10Priority (1-10)
last_firedTIMESTAMPLast execution time
next_fireTIMESTAMPNext scheduled execution
fire_countINTEGERNOT NULL DEFAULT 0Total executions
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

trigger_history

Execution log for triggers.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYHistory UUID
trigger_idTEXTNOT NULL, FK -> triggers(id) ON DELETE CASCADEParent trigger
fired_atTIMESTAMPNOT NULL DEFAULT NOW()Execution time
statusTEXTNOT NULL, CHECK IN ('success','failure','skipped')Result status
resultTEXTExecution result
errorTEXTError message
duration_msINTEGERExecution duration

plans

Autonomous multi-step plan definitions and execution state.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYPlan UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
nameTEXTNOT NULLPlan name
descriptionTEXTPlan details
goalTEXTNOT NULLWhat the plan achieves
statusTEXTNOT NULL DEFAULT 'pending', CHECK IN ('pending','running','paused','completed','failed','cancelled')Execution status
current_stepINTEGERNOT NULL DEFAULT 0Current step index
total_stepsINTEGERNOT NULL DEFAULT 0Total step count
progressREALNOT NULL DEFAULT 0, CHECK >= 0 AND <= 100Completion percent
priorityINTEGERNOT NULL DEFAULT 5, CHECK >= 1 AND <= 10Priority (1-10)
sourceTEXTWhere plan originated
source_idTEXTSource entity ID
trigger_idTEXTFK -> triggers(id) ON DELETE SET NULLAssociated trigger
goal_idTEXTFK -> goals(id) ON DELETE SET NULLAssociated goal
autonomy_levelINTEGERNOT NULL DEFAULT 1, CHECK >= 0 AND <= 4AI autonomy (0=manual, 4=full auto)
max_retriesINTEGERNOT NULL DEFAULT 3Retry limit
retry_countINTEGERNOT NULL DEFAULT 0Current retry count
timeout_msINTEGERExecution timeout
checkpointTEXTSerialized checkpoint
errorTEXTError message
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update
started_atTIMESTAMPExecution start
completed_atTIMESTAMPExecution end
metadataJSONBDEFAULT '{}'Extra metadata

plan_steps

Individual steps within a plan.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYStep UUID
plan_idTEXTNOT NULL, FK -> plans(id) ON DELETE CASCADEParent plan
order_numINTEGERNOT NULLExecution order
typeTEXTNOT NULL, CHECK IN ('tool_call','llm_decision','user_input','condition','parallel','loop','sub_plan')Step type
nameTEXTNOT NULLStep name
descriptionTEXTStep details
configJSONBNOT NULL DEFAULT '{}'Step configuration
statusTEXTNOT NULL DEFAULT 'pending', CHECK IN ('pending','running','completed','failed','skipped','blocked','waiting')Execution status
dependenciesJSONBDEFAULT '[]'Dependent step IDs
resultTEXTExecution result
errorTEXTError message
retry_countINTEGERNOT NULL DEFAULT 0Current retry count
max_retriesINTEGERNOT NULL DEFAULT 3Retry limit
timeout_msINTEGERStep timeout
started_atTIMESTAMPExecution start
completed_atTIMESTAMPExecution end
duration_msINTEGERExecution duration
on_successTEXTSuccess handler
on_failureTEXTFailure handler
metadataJSONBDEFAULT '{}'Extra metadata

plan_history

Event log for plan execution lifecycle.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYHistory UUID
plan_idTEXTNOT NULL, FK -> plans(id) ON DELETE CASCADEParent plan
step_idTEXTFK -> plan_steps(id) ON DELETE SET NULLRelated step
event_typeTEXTNOT NULL, CHECK IN ('started','step_started','step_completed','step_failed','paused','resumed','completed','failed','cancelled','checkpoint')Event type
detailsJSONBDEFAULT '{}'Event details
created_atTIMESTAMPNOT NULL DEFAULT NOW()Event time

3.5 OAuth and Media Tables

oauth_integrations

OAuth tokens for external service integrations (Gmail, Google Calendar, etc.).

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYIntegration UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
providerTEXTNOT NULLOAuth provider (google, microsoft)
serviceTEXTNOT NULLService name (gmail, calendar, drive)
access_token_encryptedTEXTNOT NULLEncrypted access token
refresh_token_encryptedTEXTEncrypted refresh token
token_ivTEXTNOT NULLInitialization vector for decryption
expires_atTIMESTAMPToken expiration
scopesJSONBNOT NULL DEFAULT '[]'Granted scopes
emailTEXTAssociated email
statusTEXTNOT NULL DEFAULT 'active', CHECK IN ('active','expired','revoked','error')Integration status
last_sync_atTIMESTAMPLast synchronization
error_messageTEXTError details
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

UNIQUE constraint on (user_id, provider, service).

media_provider_settings

Per-capability media provider routing (which provider handles image generation, TTS, etc.).

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYSetting UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
capabilityTEXTNOT NULL, CHECK IN ('image_generation','vision','tts','stt','weather')Media capability
providerTEXTNOT NULLSelected provider
modelTEXTModel identifier
configJSONBDEFAULT '{}'Provider-specific config
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

UNIQUE constraint on (user_id, capability).


3.6 AI Models Tables

user_model_configs

User overrides for AI model metadata (pricing, capabilities, display names).

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYConfig UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
provider_idTEXTNOT NULLProvider identifier
model_idTEXTNOT NULLModel identifier
display_nameTEXTCustom display name
capabilitiesJSONBNOT NULL DEFAULT '[]'Model capabilities
pricing_inputREALInput price per token
pricing_outputREALOutput price per token
context_windowINTEGERContext window size
max_outputINTEGERMax output tokens
is_enabledBOOLEANNOT NULL DEFAULT TRUEEnabled flag
is_customBOOLEANNOT NULL DEFAULT FALSECustom model flag
configJSONBDEFAULT '{}'Extra config
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

UNIQUE constraint on (user_id, provider_id, model_id).

custom_providers

User-defined AI provider aggregators (fal.ai, together.ai, etc.).

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYProvider UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
provider_idTEXTNOT NULLProvider slug
display_nameTEXTNOT NULLDisplay name
api_base_urlTEXTAPI base URL
api_key_settingTEXTSettings key for API key
provider_typeTEXTNOT NULL DEFAULT 'openai_compatible', CHECK IN ('openai_compatible','custom')Compatibility type
is_enabledBOOLEANNOT NULL DEFAULT TRUEEnabled flag
configJSONBDEFAULT '{}'Extra config
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

UNIQUE constraint on (user_id, provider_id).

user_provider_configs

User overrides for built-in AI providers (base URL, API key environment variable, enable/disable).

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYConfig UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
provider_idTEXTNOT NULLProvider slug
base_urlTEXTCustom base URL
provider_typeTEXTProvider type override
is_enabledBOOLEANNOT NULL DEFAULT TRUEEnabled flag
api_key_envTEXTEnvironment variable name
notesTEXTAdmin notes
configJSONBDEFAULT '{}'Extra config
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

UNIQUE constraint on (user_id, provider_id).


3.7 Custom Data Tables

AI-created and user-created dynamic data structures.

custom_data

Simple key-value store for AI-created dynamic data.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYEntry UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
keyTEXTNOT NULLData key
valueJSONBNOT NULLData value
metadataJSONBDEFAULT '{}'Extra metadata
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

UNIQUE constraint on (user_id, key).

custom_tools

LLM-defined or user-defined tools with executable code.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYTool UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
nameTEXTNOT NULLTool name
descriptionTEXTNOT NULLTool description
parametersJSONBNOT NULL DEFAULT '{}'JSON Schema parameters
codeTEXTNOT NULLExecutable code
categoryTEXTTool category
statusTEXTNOT NULL DEFAULT 'active', CHECK IN ('active','disabled','pending_approval','rejected')Tool status
permissionsJSONBNOT NULL DEFAULT '[]'Required permissions
requires_approvalBOOLEANNOT NULL DEFAULT FALSEApproval required flag
created_byTEXTNOT NULL DEFAULT 'user', CHECK IN ('user','llm')Creator type
versionINTEGERNOT NULL DEFAULT 1Version number
metadataJSONBDEFAULT '{}'Extra metadata
usage_countINTEGERNOT NULL DEFAULT 0Usage counter
last_used_atTIMESTAMPLast usage time
required_api_keysJSONBDEFAULT '[]'Required API key names
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

UNIQUE constraint on (user_id, name).

custom_table_schemas

Metadata for AI-created dynamic tables.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYSchema UUID
nameTEXTNOT NULL UNIQUETable name slug
display_nameTEXTNOT NULLDisplay name
descriptionTEXTTable description
columnsJSONBNOT NULL DEFAULT '[]'Column definitions
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

custom_data_records

Records stored in AI-created dynamic tables.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYRecord UUID
table_idTEXTNOT NULL, FK -> custom_table_schemas(id) ON DELETE CASCADEParent schema
dataJSONBNOT NULL DEFAULT '{}'Record data
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

3.8 Workspace Tables

Isolated user environments for code execution with Docker containers.

user_workspaces

User workspace definitions.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYWorkspace UUID
user_idTEXTNOT NULLOwner user
nameTEXTNOT NULLWorkspace name
descriptionTEXTWorkspace details
statusTEXTNOT NULL DEFAULT 'active', CHECK IN ('active','suspended','deleted')Workspace status
storage_pathTEXTNOT NULLFilesystem path
container_configJSONBNOT NULL DEFAULT '{}'Docker config
container_idTEXTActive container ID
container_statusTEXTNOT NULL DEFAULT 'stopped', CHECK IN ('stopped','starting','running','stopping','error')Container state
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update
last_activity_atTIMESTAMPLast activity

user_containers

Active Docker containers associated with workspaces.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYRecord UUID
workspace_idTEXTNOT NULL, FK -> user_workspaces(id) ON DELETE CASCADEParent workspace
user_idTEXTNOT NULLOwner user
container_idTEXTNOT NULL UNIQUEDocker container ID
imageTEXTNOT NULLDocker image
statusTEXTNOT NULL DEFAULT 'starting', CHECK IN ('stopped','starting','running','stopping','error')Container state
memory_mbINTEGERNOT NULL DEFAULT 512Memory limit
cpu_coresREALNOT NULL DEFAULT 0.5CPU core limit
network_policyTEXTNOT NULL DEFAULT 'none'Network policy
started_atTIMESTAMPNOT NULL DEFAULT NOW()Start time
last_activity_atTIMESTAMPLast activity
stopped_atTIMESTAMPStop time
memory_peak_mbINTEGERDEFAULT 0Peak memory usage
cpu_time_msINTEGERDEFAULT 0Total CPU time
network_bytes_inINTEGERDEFAULT 0Network inbound
network_bytes_outINTEGERDEFAULT 0Network outbound

code_executions

Code execution history within workspaces.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYExecution UUID
workspace_idTEXTNOT NULL, FK -> user_workspaces(id) ON DELETE CASCADEParent workspace
user_idTEXTNOT NULLOwner user
container_idTEXTContainer used
languageTEXTNOT NULL, CHECK IN ('python','javascript','shell')Code language
code_hashTEXTCode content hash
statusTEXTNOT NULL DEFAULT 'pending', CHECK IN ('pending','running','completed','failed','timeout','cancelled')Execution status
stdoutTEXTStandard output
stderrTEXTStandard error
exit_codeINTEGERProcess exit code
errorTEXTError message
execution_time_msINTEGERExecution duration
memory_used_mbINTEGERMemory consumed
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
started_atTIMESTAMPExecution start
completed_atTIMESTAMPExecution end

workspace_audit

Audit log for all workspace operations.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYAudit UUID
user_idTEXTNOT NULLActing user
workspace_idTEXTTarget workspace
actionTEXTNOT NULL, CHECK IN ('create','read','write','delete','execute','start','stop')Action type
resource_typeTEXTNOT NULL, CHECK IN ('workspace','file','container','execution')Resource type
resourceTEXTResource identifier
successBOOLEANNOT NULL DEFAULT TRUESuccess flag
errorTEXTError message
ip_addressTEXTClient IP
user_agentTEXTClient user agent
created_atTIMESTAMPNOT NULL DEFAULT NOW()Event time

3.9 Config Tables

Schema-driven service configuration management, replacing the older api_services table.

config_services

Service definitions with typed configuration schemas.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYService UUID
nameTEXTNOT NULL UNIQUEService slug
display_nameTEXTNOT NULLDisplay name
categoryTEXTNOT NULL DEFAULT 'general'Service category
descriptionTEXTService description
docs_urlTEXTDocumentation link
config_schemaJSONBNOT NULL DEFAULT '[]'Field definitions array
multi_entryBOOLEANNOT NULL DEFAULT FALSESupports multiple entries
required_byJSONBDEFAULT '[]'Tools/features that need this service
is_activeBOOLEANNOT NULL DEFAULT TRUEActive flag
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

config_entries

Actual configuration values for each service.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYEntry UUID
service_nameTEXTNOT NULLParent service name
labelTEXTNOT NULL DEFAULT 'Default'Entry label
dataJSONBNOT NULL DEFAULT '{}'Configuration values (encrypted)
is_defaultBOOLEANNOT NULL DEFAULT FALSEDefault entry flag
is_activeBOOLEANNOT NULL DEFAULT TRUEActive flag
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

Has a partial unique index: only one entry per service_name where is_default = TRUE.

Encryption at rest: data is stored as an AES-256-GCM envelope ({"__enc":"v1","iv":...,"tag":...,"ct":...}) because it holds secrets such as provider API keys. The key comes from OWNPILOT_ENCRYPTION_KEY (SHA-256-derived) or an auto-generated key file at <data>/credentials/data-encryption.key. Legacy plaintext rows are re-encrypted automatically at gateway startup. See packages/gateway/src/db/data-encryption.ts.


3.10 Plugin Tables

plugins

Plugin state persistence.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYPlugin identifier
nameTEXTNOT NULLPlugin name
versionTEXTNOT NULL DEFAULT '1.0.0'Plugin version
statusTEXTNOT NULL DEFAULT 'enabled', CHECK IN ('enabled','disabled','error')Plugin state
settingsJSONBNOT NULL DEFAULT '{}'Plugin settings
granted_permissionsJSONBNOT NULL DEFAULT '[]'Granted permissions
error_messageTEXTError details
installed_atTIMESTAMPNOT NULL DEFAULT NOW()Install time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

3.11 Local AI Tables

local_providers

Local AI inference endpoints (LM Studio, Ollama, LocalAI, vLLM).

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYProvider UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
nameTEXTNOT NULLProvider name
provider_typeTEXTNOT NULL, CHECK IN ('lmstudio','ollama','localai','vllm','custom')Provider type
base_urlTEXTNOT NULLAPI endpoint URL
api_keyTEXTOptional API key
is_enabledBOOLEANNOT NULL DEFAULT TRUEEnabled flag
is_defaultBOOLEANNOT NULL DEFAULT FALSEDefault provider flag
discovery_endpointTEXTModel list endpoint
last_discovered_atTIMESTAMPLast model discovery
metadataJSONBDEFAULT '{}'Extra metadata
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

local_models

Models available through local providers.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYModel UUID
user_idTEXTNOT NULL DEFAULT 'default'Owner user
local_provider_idTEXTNOT NULL, FK -> local_providers(id) ON DELETE CASCADEParent provider
model_idTEXTNOT NULLModel identifier
display_nameTEXTNOT NULLDisplay name
capabilitiesJSONBNOT NULL DEFAULT '["chat","streaming"]'Model capabilities
context_windowINTEGERDEFAULT 32768Context window size
max_outputINTEGERDEFAULT 4096Max output tokens
is_enabledBOOLEANNOT NULL DEFAULT TRUEEnabled flag
metadataJSONBDEFAULT '{}'Extra metadata
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

UNIQUE constraint on (user_id, local_provider_id, model_id).


3.12 Coding & CLI Tables

coding_agent_results

Persists the results of coding agent task executions.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYUUID identifier
user_idTEXTNOT NULLOwner user ID
providerTEXTNOT NULLAgent provider (claude-code, codex, etc.)
promptTEXTNOT NULLTask prompt
outputTEXTAgent output text
exit_codeINTEGERProcess exit code
duration_msINTEGERExecution duration in milliseconds
successBOOLEANDEFAULT falseWhether the task succeeded
metadataJSONBAdditional execution metadata
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation timestamp

cli_providers

Stores user-registered CLI tool providers for the coding agents system. These appear as custom:{name}.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYUUID identifier
nameTEXTNOT NULLProvider name (lowercase, alphanumeric)
display_nameTEXTNOT NULLHuman-readable display name
descriptionTEXTProvider description
binaryTEXTNOT NULLPath to binary executable
categoryTEXTTool category
iconTEXTIcon identifier
colorTEXTDisplay color
auth_methodTEXTDEFAULT 'none'none, config_center, or env_var
config_service_nameTEXTConfig Center service name
api_key_env_varTEXTEnvironment variable for API key
default_argsJSONBDefault command-line arguments
prompt_templateTEXTTemplate for prompt formatting
output_formatTEXTDEFAULT 'text'text, json, or stream-json
default_timeout_msINTEGERDefault execution timeout
max_timeout_msINTEGERMaximum allowed timeout
user_idTEXTNOT NULL DEFAULT 'default'Owner user ID
is_activeBOOLEANDEFAULT trueWhether provider is active
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation timestamp
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update timestamp

cli_tool_policies

Per-tool security policies for CLI tool execution.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYUUID identifier
user_idTEXTNOT NULLOwner user ID
tool_nameTEXTNOT NULLCLI tool name (from catalog or custom:name)
policyTEXTNOT NULLallowed, prompt, or blocked
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation timestamp
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update timestamp

Unique constraint: (user_id, tool_name) โ€” one policy per user per tool.

Repositories:

RepositorySource FileDescription
CodingAgentResultsRepositorycoding-agent-results.tsCRUD for task execution results
CliProvidersRepositorycli-providers.tsCRUD for custom CLI providers
CliToolPoliciesRepositorycli-tool-policies.tsPer-tool security policy management

3.13 Edge / IoT Tables

Migration: packages/gateway/src/db/migrations/postgres/010_edge_delegation.sql

Tables for MQTT-based IoT/edge device management, command queues, and telemetry storage.

edge_devices

Registered IoT/edge devices with sensor and actuator configurations.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYDevice UUID
user_idTEXTNOT NULLOwner user ID
nameTEXTNOT NULLDevice name
typeTEXTNOT NULL, CHECK IN (raspberry-pi, esp32, arduino, custom)Hardware type
protocolTEXTNOT NULL DEFAULT 'mqtt', CHECK IN (mqtt, websocket, http-poll)Communication protocol
sensorsJSONBNOT NULL DEFAULT '[]'Sensor configurations
actuatorsJSONBNOT NULL DEFAULT '[]'Actuator configurations
statusTEXTNOT NULL DEFAULT 'offline', CHECK IN (online, offline, error)Connection status
last_seenTIMESTAMPLast heartbeat time
firmware_versionTEXTFirmware version
metadataJSONBDEFAULT '{}'Extra metadata
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
updated_atTIMESTAMPNOT NULL DEFAULT NOW()Last update

Indexes: idx_edge_devices_user on (user_id), idx_edge_devices_status on (user_id, status).

edge_commands

Command queue for device commands sent via MQTT.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYCommand UUID
device_idTEXTNOT NULLTarget device ID
user_idTEXTNOT NULLSender user ID
command_typeTEXTNOT NULLCommand type string
payloadJSONBDEFAULT '{}'Command payload
statusTEXTNOT NULL DEFAULT 'pending', CHECK IN (pending, sent, ack, error)Command status
resultJSONBExecution result
created_atTIMESTAMPNOT NULL DEFAULT NOW()Creation time
completed_atTIMESTAMPCompletion time

Indexes: idx_edge_commands_device on (device_id), idx_edge_commands_user on (user_id).

edge_telemetry

Time-series sensor data from edge devices.

ColumnTypeConstraintsDescription
idTEXTPRIMARY KEYTelemetry UUID
device_idTEXTNOT NULLSource device
sensor_idTEXTNOT NULLSensor ID
valueJSONBNOT NULLSensor reading
recorded_atTIMESTAMPNOT NULL DEFAULT NOW()Recording time

Indexes: idx_edge_telemetry_device_sensor on (device_id, sensor_id), idx_edge_telemetry_recorded on (recorded_at DESC).

Repositories:

RepositorySource FileDescription
EdgeDevicesRepositoryedge.tsDevice CRUD, status updates, listing
EdgeCommandsRepositoryedge.tsCommand creation, status tracking, history
EdgeTelemetryRepositoryedge.tsTelemetry insertion, history queries

4. Relationships and Entity Diagram

4.1 Cascade Delete Relationships

When the parent row is deleted, all child rows are automatically removed.

conversations โ”€โ”€< messages           (ON DELETE CASCADE)
conversations โ”€โ”€< plan_history       (through plans)
channels      โ”€โ”€< channel_messages   (ON DELETE CASCADE)
plans         โ”€โ”€< plan_steps         (ON DELETE CASCADE)
plans         โ”€โ”€< plan_history       (ON DELETE CASCADE)
goals         โ”€โ”€< goal_steps         (ON DELETE CASCADE)
triggers      โ”€โ”€< trigger_history    (ON DELETE CASCADE)
habits        โ”€โ”€< habit_logs         (ON DELETE CASCADE)
custom_table_schemas โ”€โ”€< custom_data_records  (ON DELETE CASCADE)
user_workspaces โ”€โ”€< user_containers  (ON DELETE CASCADE)
user_workspaces โ”€โ”€< code_executions  (ON DELETE CASCADE)
local_providers โ”€โ”€< local_models     (ON DELETE CASCADE)

4.2 Set Null on Delete Relationships

When the parent row is deleted, the foreign key in the child is set to NULL.

conversations  <โ”€โ”€ request_logs.conversation_id  (ON DELETE SET NULL)
conversations  <โ”€โ”€ costs.conversation_id         (ON DELETE SET NULL)
triggers       <โ”€โ”€ plans.trigger_id              (ON DELETE SET NULL)
goals          <โ”€โ”€ plans.goal_id                 (ON DELETE SET NULL)
plan_steps     <โ”€โ”€ plan_history.step_id          (ON DELETE SET NULL)

4.3 Self-Referencing Relationships

tasks.parent_id  -> tasks.id    (ON DELETE SET NULL)  -- subtask hierarchy
goals.parent_id  -> goals.id    (ON DELETE SET NULL)  -- sub-goal hierarchy

4.4 Conceptual Entity Relationship Diagram

                          CORE
 โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
 โ”‚                                                          โ”‚
 โ”‚  settings (key-value)         agents                     โ”‚
 โ”‚                                                          โ”‚
 โ”‚  conversations โ”€โ”€โ”€โ”€โ”€โ”€< messages                          โ”‚
 โ”‚       โ”‚                                                  โ”‚
 โ”‚       โ”œโ”€โ”€โ”€ request_logs (SET NULL)                       โ”‚
 โ”‚       โ””โ”€โ”€โ”€ costs (SET NULL)                              โ”‚
 โ”‚                                                          โ”‚
 โ”‚  channels โ”€โ”€โ”€โ”€โ”€โ”€< channel_messages                       โ”‚
 โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

                     PERSONAL DATA
 โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
 โ”‚                                                          โ”‚
 โ”‚  bookmarks    notes    contacts    reminders  captures   โ”‚
 โ”‚                                                          โ”‚
 โ”‚  projects ยทยทยทยท tasks.project_id (logical)                โ”‚
 โ”‚                                                          โ”‚
 โ”‚  tasks โ”€โ”€โ”  (self-ref: parent_id)                        โ”‚
 โ”‚          โ””โ”€โ”€> tasks                                      โ”‚
 โ”‚                                                          โ”‚
 โ”‚  calendar_events                                         โ”‚
 โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

                      PRODUCTIVITY
 โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
 โ”‚                                                          โ”‚
 โ”‚  pomodoro_settings    pomodoro_sessions                  โ”‚
 โ”‚                       pomodoro_daily_stats               โ”‚
 โ”‚                                                          โ”‚
 โ”‚  habits โ”€โ”€โ”€โ”€โ”€โ”€< habit_logs                               โ”‚
 โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

                     AUTONOMOUS AI
 โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
 โ”‚                                                          โ”‚
 โ”‚  memories                                                โ”‚
 โ”‚                                                          โ”‚
 โ”‚  goals โ”€โ”€โ”  (self-ref: parent_id)                        โ”‚
 โ”‚          โ””โ”€โ”€> goals                                      โ”‚
 โ”‚       โ””โ”€โ”€โ”€โ”€โ”€โ”€< goal_steps                                โ”‚
 โ”‚                                                          โ”‚
 โ”‚  triggers โ”€โ”€โ”€โ”€โ”€โ”€< trigger_history                        โ”‚
 โ”‚       โ”‚                                                  โ”‚
 โ”‚       โ””โ”€โ”€ plans.trigger_id (SET NULL)                    โ”‚
 โ”‚                                                          โ”‚
 โ”‚  goals โ”€โ”€โ”€ plans.goal_id (SET NULL)                      โ”‚
 โ”‚                                                          โ”‚
 โ”‚  plans โ”€โ”€โ”€โ”€โ”€โ”€< plan_steps                                โ”‚
 โ”‚       โ””โ”€โ”€โ”€โ”€โ”€โ”€< plan_history ยทยทยทยท plan_steps (SET NULL)   โ”‚
 โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

                       EDGE / IoT
 โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
 โ”‚                                                          โ”‚
 โ”‚  edge_devices โ”€โ”€โ”€โ”€โ”€โ”€< edge_commands                      โ”‚
 โ”‚               โ”€โ”€โ”€โ”€โ”€โ”€< edge_telemetry                     โ”‚
 โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

                   WORKSPACE / CONFIG / AI
 โ”Œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”
 โ”‚                                                          โ”‚
 โ”‚  user_workspaces โ”€โ”€< user_containers                     โ”‚
 โ”‚                  โ”€โ”€< code_executions                     โ”‚
 โ”‚  workspace_audit                                         โ”‚
 โ”‚                                                          โ”‚
 โ”‚  oauth_integrations    media_provider_settings           โ”‚
 โ”‚                                                          โ”‚
 โ”‚  user_model_configs    custom_providers                  โ”‚
 โ”‚  user_provider_configs                                   โ”‚
 โ”‚                                                          โ”‚
 โ”‚  config_services    config_entries                        โ”‚
 โ”‚                                                          โ”‚
 โ”‚  plugins                                                 โ”‚
 โ”‚                                                          โ”‚
 โ”‚  local_providers โ”€โ”€< local_models                        โ”‚
 โ”‚                                                          โ”‚
 โ”‚  custom_data     custom_tools                            โ”‚
 โ”‚  custom_table_schemas โ”€โ”€< custom_data_records            โ”‚
 โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜

5. Index Strategy

All indexes use CREATE INDEX IF NOT EXISTS for idempotent execution. The strategy follows these principles:

5.1 Access Pattern Indexes

Every table with a user_id column has an index on (user_id) to support multi-tenant filtering:

CREATE INDEX IF NOT EXISTS idx_conversations_user ON conversations(user_id);
CREATE INDEX IF NOT EXISTS idx_tasks_user ON tasks(user_id);
-- ... repeated for all user-scoped tables

5.2 Status and Filter Indexes

Tables with status, type, category, or boolean flag columns have single-column indexes to accelerate WHERE clauses:

CREATE INDEX IF NOT EXISTS idx_tasks_status ON tasks(status);
CREATE INDEX IF NOT EXISTS idx_triggers_enabled ON triggers(enabled);
CREATE INDEX IF NOT EXISTS idx_habits_archived ON habits(is_archived);

5.3 Temporal Indexes (DESC)

Frequently sorted-by-time columns use descending indexes:

CREATE INDEX IF NOT EXISTS idx_conversations_updated ON conversations(updated_at DESC);
CREATE INDEX IF NOT EXISTS idx_request_logs_created ON request_logs(created_at DESC);
CREATE INDEX IF NOT EXISTS idx_memories_importance ON memories(importance DESC);

5.4 Foreign Key Indexes

Every foreign key column is indexed to accelerate JOIN operations and cascade deletes:

CREATE INDEX IF NOT EXISTS idx_messages_conversation ON messages(conversation_id);
CREATE INDEX IF NOT EXISTS idx_plan_steps_plan ON plan_steps(plan_id);
CREATE INDEX IF NOT EXISTS idx_goal_steps_goal ON goal_steps(goal_id);

5.5 Composite Indexes

Multi-column indexes for common compound lookups:

CREATE INDEX IF NOT EXISTS idx_plan_steps_order ON plan_steps(plan_id, order_num);
CREATE INDEX IF NOT EXISTS idx_oauth_integrations_service ON oauth_integrations(user_id, provider, service);
CREATE INDEX IF NOT EXISTS idx_custom_data_key ON custom_data(user_id, key);
CREATE INDEX IF NOT EXISTS idx_habit_logs_user_date ON habit_logs(user_id, date);
CREATE INDEX IF NOT EXISTS idx_pomodoro_daily_user_date ON pomodoro_daily_stats(user_id, date);

5.6 Partial Unique Index

The config_entries table has a partial unique index ensuring only one default entry per service:

CREATE UNIQUE INDEX IF NOT EXISTS idx_config_entries_default
  ON config_entries(service_name) WHERE is_default = TRUE;

5.7 Total Index Count by Group

GroupIndex Count
Core (conversations, messages, request_logs, channels, costs)17
Personal Data (bookmarks, notes, tasks, calendar, contacts, projects, reminders, captures)15
Productivity (pomodoro, habits)10
Autonomous AI (memories, goals, triggers, plans)22
Workspace9
OAuth and Media6
AI Models9
Custom Data and Tools7
Config Services3 (created inline in MIGRATIONS_SQL)
Local AI4
Total102+

6. Repository Pattern

Source directory: packages/gateway/src/db/repositories/

Every data domain has a dedicated repository class that extends BaseRepository. Repositories encapsulate all SQL queries and expose typed methods. There are 40+ repository files.

6.1 BaseRepository

File: packages/gateway/src/db/repositories/base.ts

export abstract class BaseRepository {
  protected adapter: DatabaseAdapter | null = null;

  // Core query methods
  protected async query<T>(sql: string, params?: unknown[]): Promise<T[]>;
  protected async queryOne<T>(sql: string, params?: unknown[]): Promise<T | null>;
  protected async execute(sql: string, params?: unknown[]): Promise<{ changes: number }>;
  protected async exec(sql: string): Promise<void>;
  protected async transaction<T>(fn: () => Promise<T>): Promise<T>;

  // SQL dialect helpers
  protected now(): string; // Returns 'NOW()'
  protected boolean(value: boolean): unknown;
  protected parseBoolean(value: unknown): boolean;
}

The BaseRepository lazily acquires the database adapter via getAdapter(). All repositories inherit this behavior.

6.2 IRepository Interface

All repositories conform to the IRepository<T> interface pattern, which standardizes common query operations:

interface IRepository<T> {
  getById(id: string): Promise<T | null>;
  list(query?: StandardQuery): Promise<T[]>;
  create(data: Partial<T>): Promise<T>;
  update(id: string, data: Partial<T>): Promise<T | null>;
  delete(id: string): Promise<boolean>;
}

interface StandardQuery {
  limit?: number;
  offset?: number;
  orderBy?: string;
  orderDir?: 'asc' | 'desc';
  search?: string;
  filters?: Record<string, unknown>;
}

interface PaginatedResult<T> {
  items: T[];
  total: number;
  page: number;
  pageSize: number;
  totalPages: number;
}

The BaseRepository provides a paginatedQuery() helper that wraps standard list queries with pagination:

protected async paginatedQuery<T>(
  sql: string,
  countSql: string,
  params: unknown[],
  page: number,
  pageSize: number
): Promise<PaginatedResult<T>>;

6.3 Repository Catalog

Each repository exports: the repository class, a factory function, TypeScript interfaces for row types and input types, and optionally a singleton instance.

Repository FileClassTables Managed
conversations.tsConversationsRepositoryconversations
messages.tsMessagesRepositorymessages
chat.tsChatRepositoryconversations + messages (combined)
logs.tsLogsRepositoryrequest_logs
channels.tsChannelsRepositorychannels
channel-messages.tsChannelMessagesRepositorychannel_messages
costs.tsCostsRepositorycosts
agents.tsAgentsRepositoryagents
settings.tsSettingsRepositorysettings
bookmarks.tsBookmarksRepositorybookmarks
notes.tsNotesRepositorynotes
tasks.tsTasksRepositorytasks
calendar.tsCalendarRepositorycalendar_events
contacts.tsContactsRepositorycontacts
captures.tsCapturesRepositorycaptures
pomodoro.tsPomodoroRepositorypomodoro_sessions, pomodoro_settings, pomodoro_daily_stats
habits.tsHabitsRepositoryhabits, habit_logs
memories.tsMemoriesRepositorymemories
goals.tsGoalsRepositorygoals, goal_steps
triggers.tsTriggersRepositorytriggers, trigger_history
plans.tsPlansRepositoryplans, plan_steps, plan_history
oauth-integrations.tsOAuthIntegrationsRepositoryoauth_integrations
media-settings.tsMediaSettingsRepositorymedia_provider_settings
model-configs.tsModelConfigsRepositoryuser_model_configs, custom_providers, user_provider_configs
custom-data.tsCustomDataRepositorycustom_data, custom_table_schemas, custom_data_records
custom-tools.tsCustomToolsRepositorycustom_tools
workspaces.tsWorkspacesRepositoryuser_workspaces, user_containers, code_executions, workspace_audit
plugins.tsPluginsRepositoryplugins
config-services.tsConfigServicesRepositoryconfig_services, config_entries
local-providers.tsLocalProvidersRepositorylocal_providers, local_models

6.4 Usage Pattern

Repositories are consumed in two ways:

Factory function (for user-scoped data):

import { createTasksRepository } from './repositories/index.js';

const tasksRepo = createTasksRepository('user-123');
const tasks = await tasksRepo.list({ status: 'pending' });

Singleton instance (for global data):

import { agentsRepo, settingsRepo } from './repositories/index.js';

const agents = await agentsRepo.getAll();
const theme = await settingsRepo.get('theme');

6.5 Repository Index

File: packages/gateway/src/db/repositories/index.ts

Re-exports all repository classes, factory functions, singleton instances, and TypeScript type definitions from every repository file. This is the single import point for consumers.


7. Data Stores

File: packages/gateway/src/db/data-stores.ts

Data stores implement the DataStore<T> interface from @ownpilot/core, providing a clean abstraction layer between the core package and PostgreSQL repositories. Each store wraps a repository and handles type mapping (database row types to core domain types).

7.1 Available Stores

Store ClassWraps RepositoryCore Type
BookmarkStoreBookmarksRepositoryBookmark
NoteStoreNotesRepositoryNote
TaskStoreTasksRepositoryTask
CalendarStoreCalendarRepositoryPersonalCalendarEvent
ContactStoreContactsRepositoryContact

7.2 DataStore Interface

Each store implements these methods:

interface DataStore<T> {
  get(id: string): Promise<T | null>;
  list(filter?: Record<string, unknown>): Promise<T[]>;
  search(query: string): Promise<T[]>;
  create(data: Omit<T, 'id' | 'createdAt'>): Promise<T>;
  update(id: string, data: Partial<T>): Promise<T | null>;
  delete(id: string): Promise<boolean>;
}

7.3 Factory Function

import { createDataStores } from './data-stores.js';

const stores = createDataStores('user-123');
// stores.bookmarks, stores.notes, stores.tasks, stores.calendar, stores.contacts

7.4 Backwards Compatibility

Legacy aliases are exported for migration from the older SQLite-based system:

export const SQLiteBookmarkStore = BookmarkStore; // @deprecated
export const SQLiteNoteStore = NoteStore; // @deprecated
export const SQLiteTaskStore = TaskStore; // @deprecated
export const SQLiteCalendarStore = CalendarStore; // @deprecated
export const SQLiteContactStore = ContactStore; // @deprecated
export const createSQLiteDataStores = createDataStores; // @deprecated

8. Seed Data

8.1 Default Agents

File: packages/gateway/src/db/seeds/default-agents.ts

Loads agent configurations from the JSON file at packages/gateway/data/seeds/default-agents.json. Each agent includes:

  • id and name (with optional emoji prefix)
  • systemPrompt -- the agent's personality and instructions
  • provider / model -- always set to 'default' (resolved at runtime)
  • config -- includes maxTokens, temperature, maxTurns, maxToolCalls, tools, and toolGroups

The seedDefaultAgents() function (in seeds/index.ts) only seeds when the agents table is empty. It uses the agentsRepo singleton to insert each agent.

8.2 Example Plans

File: packages/gateway/src/db/seeds/plans-seed.ts

Creates three example plans with steps:

Plan NameStepsPurpose
Weekly Goal Review3 (tool_call, tool_call, llm_decision)Review active goals and generate insights
Daily Memory Digest2 (tool_call, llm_decision)Summarize recent memories
Task Cleanup2 (tool_call, llm_decision)Find overdue or stale tasks

The seedExamplePlans() function checks existing plans by name and skips duplicates.

8.3 Config Services

File: packages/gateway/src/db/seeds/config-services-seed.ts

Pre-populates the config_services table with definitions for known external services. Each entry defines:

  • A typed configSchema (array of field definitions with types like secret, string, url, number, boolean, select)
  • Category classification (weather, email, media, translation, search, messaging)
  • Documentation URLs

Known services include: OpenWeatherMap, WeatherAPI, SMTP, IMAP, ElevenLabs, Deepgram, DeepL, Tavily, Serper, Perplexity, and Telegram.

The seedConfigServices() function uses idempotent upserts and cleans up stale services that are no longer in the seed list.

8.4 Main Seed Script

File: packages/gateway/scripts/seed-database.ts

A standalone script that seeds data via the REST API (requires the server to be running):

npx tsx scripts/seed-database.ts

Seeds: agents (from JSON), sample tasks, sample notes, sample memories, and sample goals.

8.5 Trigger and Plan Seeds

File: packages/gateway/scripts/seed-triggers-plans.ts

A standalone script that seeds trigger and plan data via the REST API:

npx tsx scripts/seed-triggers-plans.ts

Seeds 10 sample triggers across all four types (schedule, event, condition, webhook) and 5 sample plans with full step definitions (Morning Routine Analysis, Weekly Goal Review, Email Processing Pipeline, Code Review Assistant, Research Topic Deep Dive).


9. Migrations

9.1 Schema Migrations (MIGRATIONS_SQL)

All schema migrations live in the MIGRATIONS_SQL constant in schema.ts. They follow a strict idempotent pattern using PL/pgSQL:

DO $$ BEGIN
  IF NOT EXISTS (
    SELECT 1 FROM information_schema.columns
    WHERE table_name = 'table_name' AND column_name = 'column_name'
  ) THEN
    ALTER TABLE table_name ADD COLUMN column_name TYPE DEFAULT value;
  END IF;
END $$;

This pattern is safe to run any number of times. The migrations cover:

triggers table:

  • Added enabled column (BOOLEAN DEFAULT TRUE)

custom_tools table (major migration):

  • implementation -> code (column rename with data migration)
  • enabled -> status (boolean to enum migration)
  • source -> created_by (column rename with data migration)
  • Added: category, permissions, requires_approval, version, metadata, usage_count, last_used_at, required_api_keys

Model and provider config tables:

  • Added is_enabled column to user_model_configs, custom_providers, user_provider_configs

conversations table:

  • Added agent_name column

request_logs table:

  • Added error_stack column

9.2 Table Migration (api_services -> config_services)

The most complex migration replaces the old api_services table with config_services + config_entries:

  1. Creates new config_services and config_entries tables
  2. Checks if api_services exists
  3. Migrates service definitions with schema transformation
  4. Migrates configuration values (api_key, base_url, extra_config) into config_entries
  5. Drops the old api_services table

9.3 SQLite to PostgreSQL Migration

File: packages/gateway/scripts/migrate-to-postgres.ts

A full database migration tool for users moving from the deprecated SQLite backend:

# Preview what would be migrated
pnpm tsx scripts/migrate-to-postgres.ts --dry-run

# Migrate all data
pnpm tsx scripts/migrate-to-postgres.ts

# Clear target tables first, then migrate
pnpm tsx scripts/migrate-to-postgres.ts --truncate

# Skip schema creation (if already exists)
pnpm tsx scripts/migrate-to-postgres.ts --skip-schema

The migrator:

  1. Opens the SQLite database in read-only mode
  2. Connects to PostgreSQL via a connection pool
  3. Iterates tables in dependency order (26 tables, parents before children)
  4. Converts SQLite types: INTEGER booleans -> true/false, TEXT timestamps -> TIMESTAMP, TEXT JSON -> JSONB
  5. Inserts rows in batches of 100 with ON CONFLICT DO NOTHING
  6. Prints a detailed summary table with success/failure counts

10. Connection Management

10.1 Adapter Architecture

Files: packages/gateway/src/db/adapters/

The database layer uses an adapter pattern with a single implementation:

DatabaseAdapter (interface)
  โ””โ”€โ”€ PostgresAdapter (pg Pool)

The DatabaseAdapter interface defines:

MethodDescription
query<T>(sql, params)Execute SELECT, return rows
queryOne<T>(sql, params)Execute SELECT, return first row or null
execute(sql, params)Execute INSERT/UPDATE/DELETE, return change count
transaction<T>(fn)Execute function in BEGIN/COMMIT/ROLLBACK
exec(sql)Execute raw SQL (schema changes)
close()Close connection pool
now()SQL timestamp expression (NOW())
placeholder(index)Parameter placeholder ($1, $2, ...)
boolean(value)Convert boolean for storage

10.2 PostgreSQL Connection Pool

The PostgresAdapter uses the pg library (node-postgres) with connection pooling:

this.pool = new Pool({
  connectionString: config.postgresUrl,
  max: config.postgresPoolSize || 10, // Maximum connections
  idleTimeoutMillis: 30000, // Close idle connections after 30s
  connectionTimeoutMillis: 5000, // Timeout on new connection attempts
});

10.3 Global Singleton

The adapter module (adapters/index.ts) maintains a global singleton:

// First call creates the adapter and initializes the schema
const adapter = await getAdapter();

// Subsequent calls return the cached instance
const adapter = await getAdapter();

// Synchronous access (must be initialized first)
const adapter = getAdapterSync();

// Explicit initialization
await initializeAdapter(config);

// Cleanup on shutdown
await closeAdapter();

10.4 Schema Auto-Initialization

On the first createAdapter() call, the schema is automatically initialized:

if (!schemaInitialized) {
  await initializeSchema(async (sql) => pgAdapter.exec(sql));
  schemaInitialized = true;
}

This ensures the database schema is always up to date when the application starts.

10.5 Placeholder Conversion

The adapter automatically converts SQLite-style ? placeholders to PostgreSQL $1, \$2, ... style, allowing repositories to use either format:

private convertPlaceholders(sql: string): string {
  let index = 0;
  return sql.replace(/\?/g, () => {
    index++;
    return `$${index}`;
  });
}

11. Docker Compose Setup

11.1 Database-Only (Development)

File: docker-compose.db.yml

Runs only the PostgreSQL container for local development:

docker compose -f docker-compose.db.yml up -d

Configuration:

ParameterDefaultDescription
Imagepostgres:16-alpinePostgreSQL 16 on Alpine
Container nameownpilot-dbDocker container name
Host port25432Maps to container port 5432
UsernameownpilotPostgreSQL user
Passwordownpilot_secretPostgreSQL password
DatabaseownpilotPostgreSQL database name
Volumeownpilot-postgres-dataPersistent data

Health check: pg_isready -U ownpilot -d ownpilot (every 10s, 5 retries).

11.2 Full Stack

File: docker-compose.yml

Runs PostgreSQL, the gateway API server, and optionally the UI:

# Full stack with database
docker compose --profile postgres up -d

# Full stack with UI
docker compose --profile postgres --profile ui up -d

The gateway service depends on the PostgreSQL service with a health check condition. Environment variables are passed through for database connection, API keys, Telegram integration, authentication, and autonomy settings.


12. Environment Variables

12.1 Database Connection

VariableRequiredDefaultDescription
DATABASE_URLNo(built from parts below)Full PostgreSQL connection string
POSTGRES_HOSTNolocalhostPostgreSQL hostname
POSTGRES_PORTNo25432PostgreSQL port
POSTGRES_USERNoownpilotPostgreSQL username
POSTGRES_PASSWORDNoownpilot_secretPostgreSQL password
POSTGRES_DBNoownpilotPostgreSQL database name
POSTGRES_POOL_SIZENo10Maximum connection pool size
DB_VERBOSENofalseEnable verbose SQL logging

If DATABASE_URL is not set, the connection string is built from the individual POSTGRES_* variables:

postgresql://{POSTGRES_USER}:{POSTGRES_PASSWORD}@{POSTGRES_HOST}:{POSTGRES_PORT}/{POSTGRES_DB}

12.2 Production Warning

In production (NODE_ENV=production), the system logs a warning if no explicit database credentials are provided. Default credentials are intended only for local development with Docker Compose.

12.3 Default Connection String

For local development with the provided Docker Compose setup:

postgresql://ownpilot:ownpilot_secret@localhost:25432/ownpilot