AI Output Storage Backend - Data Models

August 9, 2025 · View on GitHub

A comprehensive data structure for storing and managing AI outputs, prompts, and related metadata in N8N workflows.

🏗️ Architecture Overview

The data model is organized into three primary domains:

1. OUTPUTS Domain 📤

Tables related to AI-generated content and their lifecycle management.

2. INFERENCE Domain 🤖

Tables managing AI agents, assistants, models, and inference infrastructure.

3. USERS Domain 👥

Tables handling user interactions, prompts, sessions, and authentication.

📊 Data Model Diagrams

OUTPUTS Domain - Comprehensive Output Lifecycle Management

graph TD
    O[outputs] --> OL[output_lifecycle]
    O --> OR[output_revisions]
    O --> OUA[output_usage_analytics]
    O --> OKE[output_knowledge_extraction]
    O --> OSR[output_semantic_relationships]
    O --> OVA[output_value_assessment]
    O --> OC[output_collections]
    
    O --> OT[output_tracking]
    O --> QA[quality_assessments]
    O --> BF[binary_file_data]
    O --> PII[pii_detection]
    
    QA --> QAT[qa_validation_taxonomy]
    O --> IS[information_sensitivity]
    IS --> ISP[information_sharing_policy]
    
    O --> RP[retention_policies]
    RP --> DRP[data_retention_policies]
    
    IFL[improvement_feedback_loop] --> O
    IFL --> AG[ai_agents]
    
    style O fill:#e1f5fe
    style OL fill:#e8f5e8
    style OR fill:#e8f5e8
    style OUA fill:#e8f5e8
    style QA fill:#f3e5f5
    style IS fill:#fff3e0

INFERENCE Domain

graph TD
    AA[ai_assistants] --> SP[system_prompts]
    AA --> LLM[llm_models]
    AA --> STT[speech_to_text_models]
    
    AG[ai_agents] --> NW[n8n_workflows]
    
    API[api_usage] --> AP[api_parameters]
    API --> UI[ui_interfaces]
    
    MCP[mcp_usage]
    
    AA --> API
    AG --> API
    
    style AA fill:#e8f5e8
    style AG fill:#e8f5e8
    style LLM fill:#fff8e1
    style API fill:#fce4ec

USERS Domain

graph TD
    U[users] --> S[sessions]
    S --> C[conversations]
    C --> P[prompts]
    P --> PT[prompt_tracking]
    
    U --> CR[credentials]
    P --> BF[binary_file_data]
    P --> PII[pii_detection]
    
    style U fill:#e3f2fd
    style S fill:#e3f2fd
    style C fill:#e3f2fd
    style P fill:#e1f5fe

Cross-Domain Relationships

graph LR
    subgraph USERS
        U[users]
        P[prompts]
        S[sessions]
    end
    
    subgraph INFERENCE
        AA[ai_assistants]
        LLM[llm_models]
        API[api_usage]
    end
    
    subgraph OUTPUTS
        O[outputs]
        QA[quality_assessments]
        IS[information_sensitivity]
    end
    
    P --> O
    P --> API
    AA --> O
    LLM --> API
    O --> QA
    O --> IS
    
    style USERS fill:#e3f2fd,stroke:#1976d2
    style INFERENCE fill:#e8f5e8,stroke:#388e3c
    style OUTPUTS fill:#fff3e0,stroke:#f57c00

Core Tables - Primary Data Storage

Table	Description	View	Raw CSV
Prompts	User prompts with delivery methods & feedback	📄	📊
Outputs	AI responses with quality metrics & revisions	📄	📊
Conversations	Session grouping for prompts & outputs	📄	📊
Binary File Data	File storage with encryption & metadata	📄	📊

System Tables - AI Infrastructure Management

Table	Description	View	Raw CSV
AI Assistants	Assistant definitions with performance metrics	📄	📊
System Prompts	System prompt versioning & management	📄	📊
AI Agents	N8N workflow agents with feedback tracking	📄	📊
N8N Workflows	Workflow execution & performance tracking	📄	📊
Users	User profiles & subscription management	📄	📊
Sessions	User interaction session tracking	📄	📊
Credentials	API credential management	📄	📊

Tracking Tables - Comprehensive Lifecycle Management

Table	Description	View	Raw CSV
Output Lifecycle	Stage-by-stage output progression	📄	📊
Output Revisions	Version control & improvement history	📄	📊
Output Usage Analytics	Real-world usage & effectiveness tracking	📄	📊
Output Knowledge Extraction	Insights & actionable items extraction	📄	📊
Output Semantic Relationships	Inter-output connection mapping	📄	📊
Output Value Assessment	Business & educational value scoring	📄	📊
Output Collections	Curated high-value output collections	📄	📊
Agent Performance Metrics	Daily agent performance tracking	📄	📊
Improvement Feedback Loop	Systematic improvement tracking	📄	📊
Output Tracking	Task management for outputs	📄	📊
Prompt Tracking	Task management for prompts	📄	📊
Quality Assessments	Output quality evaluation	📄	📊

Lookup Tables - Reference Data & Configuration

Table	Description	View	Raw CSV
LLM Models	Model definitions with costs & capabilities	📄	📊
Speech-to-Text Models	STT models with accuracy ratings	📄	📊
UI Interfaces	Interface definitions & capabilities	📄	📊
API Parameters	API parameter specifications	📄	📊
QA Validation Taxonomy	Quality assessment criteria & weights	📄	📊
Data Retention Policies	Enhanced retention rules & conditions	📄	📊
API Usage	API call tracking & cost monitoring	📄	📊
MCP Usage	Model Context Protocol tool usage	📄	📊
Retention Policies	Basic retention policy definitions	📄	📊

Security Tables - Compliance & Privacy Management

Table	Description	View	Raw CSV
PII Detection	Personal information tracking & masking	📄	📊
Information Sensitivity	4-tier sensitivity classification	📄	📊
Information Sharing Policy	Sharing rules by sensitivity level	📄	📊

Documentation

Document	Description	View
Data Model Overview	Comprehensive technical documentation	📖

📁 Directory Structure

├── core-tables/           # Primary data storage
│   ├── prompts.csv
│   ├── outputs.csv
│   ├── conversations.csv
│   └── binary_file_data.csv
├── system-tables/         # AI system management
│   ├── ai_assistants.csv
│   ├── system_prompts.csv
│   ├── ai_agents.csv
│   ├── n8n_workflows.csv
│   ├── users.csv
│   ├── sessions.csv
│   └── credentials.csv
├── tracking-tables/       # Comprehensive output lifecycle & improvement
│   ├── output_tracking.csv
│   ├── prompt_tracking.csv
│   ├── quality_assessments.csv
│   ├── output_lifecycle.csv
│   ├── output_revisions.csv
│   ├── output_usage_analytics.csv
│   ├── output_knowledge_extraction.csv
│   ├── output_semantic_relationships.csv
│   ├── output_value_assessment.csv
│   ├── output_collections.csv
│   ├── agent_performance_metrics.csv
│   └── improvement_feedback_loop.csv
├── lookup-tables/         # Reference data
│   ├── llm_models.csv
│   ├── speech_to_text_models.csv
│   ├── ui_interfaces.csv
│   ├── api_parameters.csv
│   ├── qa_validation_taxonomy.csv
│   ├── retention_policies.csv
│   ├── data_retention_policies.csv
│   ├── api_usage.csv
│   └── mcp_usage.csv
├── security-tables/       # Security & compliance
│   ├── pii_detection.csv
│   ├── information_sensitivity.csv
│   └── information_sharing_policy.csv
└── docs/                  # Documentation
    └── data-model-overview.md