Chatbook API Documentation

Overview

The Chatbook API provides functionality for exporting and importing full user-account backups or explicit content allowlists in a portable archive format. It also supports importing OpenWebUI chat export JSON and uploaded OpenWebUI webui.db database files through the same import and preview endpoints. This enables users to backup, share, and migrate their data between instances or users.

Developer Code Guide: Docs/Code_Documentation/Guides/Chatbooks_Code_Guide.md:1

Auth + Rate Limits

• Single-user: X-API-KEY: <key>
• Multi-user: Authorization: Bearer <JWT>
• Standard limits apply; export/import run as background jobs and may be constrained by per-user concurrency.

Table of Contents

1. Introduction
2. Authentication
3. API Endpoints
4. Data Models
5. Error Handling
6. Rate Limiting
7. Code Examples
8. Migration Guide

Introduction

What is a Chatbook?

A Chatbook is a portable archive format (.chatbook file) that contains:

• Conversations and chat histories
• Notes and documents
• Character definitions
• World books and lore
• Dictionaries for text replacement
• Generated documents
• Media records, transcripts, chunks, derived metadata, and tldw-stored account file artifacts
• Embeddings and vector/search data when present
• Account settings, prompts, evaluations, relationships, and sensitive account values required for restore

Full-account export scope is defined by the Chatbooks account-data inventory. Stored account file artifacts are bundled under content/media/files/*. External URLs, local source paths, and other provenance pointers are exported as pointer metadata only unless tldw owns stored artifact bytes for that account.

Key Features

• Full Account Backup: Omit content_selections or pass {} to export the full user-account inventory
• Selective Export: Pass a non-empty content_selections object to export an explicit allowlist
• Conflict Resolution: Multiple strategies for handling duplicate content during import
• Async Processing: Large operations can run in the background
• Job Management: Track progress of export/import operations
• Security: File validation, size limits, and path traversal protection
• User Isolation: Content is automatically scoped to authenticated users

Request Flow (ASCII)

Export (sync)
  Client → POST /api/v1/chatbooks/export (async_mode=false)
         → Validate + Quotas → Service writes manifest+files → ZIP → ExportJob completed
         ← 200 { job_id, download_url }

Export (async)
  Client → POST /api/v1/chatbooks/export (async_mode=true)
         → ExportJob pending → enqueue core Jobs (domain=chatbooks) or Prompt Studio
         ← 200 { job_id }
  Worker → process → write ZIP → update job (download_url, expires_at, status)

Import (sync/async)
  Client → POST /api/v1/chatbooks/import (multipart)
         → Save temp → Branch by source_format
         → Chatbook archive: Validate ZIP → Secure extract → Restore restorable archive account data
         → OpenWebUI JSON: Parse export JSON → Import chat trees
         → OpenWebUI DB: Preview users → Require selected user → Import that user's chat trees and mirror folders
         ← Sync: { success, imported_items, warnings } or Async: { job_id }

OpenWebUI attachment hydration
  Client → POST /api/v1/chatbooks/openwebui/hydration/preview (JSON)
         → Validate owner/admin access → Resolve data root under allowed roots
         → Read selected import scope or manual conversation IDs + webui.db file rows → Return counts/warnings
  Client → POST /api/v1/chatbooks/openwebui/hydration/jobs (JSON)
         → Enqueue openwebui_attachment_hydration job
  Worker → Copy referenced images, register supported files, optionally process files

Authentication

Supported modes (server decides based on AUTH_MODE):

• Single-user mode: X-API-KEY: <key>
• Multi-user mode: Authorization: Bearer <JWT>

Examples:

# Single-user
X-API-KEY: <your-api-key>

# Multi-user
Authorization: Bearer <your-jwt-token>

API Endpoints

1. Create Chatbook Export

Endpoint: POST /api/v1/chatbooks/export

Description: Create a new chatbook export. Omit content_selections or send {} for a full user-account backup; send a non-empty object for explicit allowlist export.

Full Account Request Body:

{
  "name": "Weekly Backup - January 2024",
  "description": "Complete backup of all restorable account data",
  "include_media": true,
  "media_quality": "original",
  "include_embeddings": true,
  "include_generated_content": true,
  "format_version": "1.1.0",
  "tags": ["backup", "weekly"],
  "categories": ["work"],
  "async_mode": true
}

"content_selections": {} is equivalent to omitting the field and also means full user-account export. This compatibility behavior is intentional.

Selective Request Body:

{
  "name": "Project Alpha Archive",
  "description": "Selected conversations and notes",
  "content_selections": {
    "conversation": ["conv_123", "conv_456"],
    "note": ["note_789"],
    "character": []
  },
  "author": "John Doe",
  "include_media": true,
  "media_quality": "compressed",
  "include_embeddings": false,
  "include_generated_content": true,
  "format_version": "1.0.0",
  "tags": ["backup", "weekly"],
  "categories": ["work"],
  "async_mode": false
}

When content_selections is non-empty, it is an explicit allowlist. Empty arrays in that allowlist mean no items for that type, not "all items." A non-empty allowlist that resolves to zero exportable items returns a 4xx validation error.

format_version is optional and defaults to "1.0.0". Chatbook v1.1 export is opt-in: clients must request "format_version": "1.1.0" to receive the v1.1 manifest additions, file inventory, account inventory summary, and any v1.1 content envelopes. If the field is omitted, export remains v1.0.0/legacy-compatible.

Response (Synchronous):

{
  "success": true,
  "message": "Chatbook created successfully",
  "job_id": "0c9d9a3a-6d1c-4c8f-9c84-9a0c2c2d8f77",
  "download_url": "/api/v1/chatbooks/download/0c9d9a3a-6d1c-4c8f-9c84-9a0c2c2d8f77"
}

Implementation notes:

• Sync mode persists a completed export job and returns its job_id plus a download_url that uses this UUID.
• For robust automation, prefer async mode and then poll job status to obtain the canonical download_url by job_id.
• Completed full-account jobs expose redacted inventory metadata such as archive size, pointer-only count, warning count, and post-write verification status. Secret values are not returned in preview, manifest summaries, job summaries, logs, or audit metadata.
• When evaluation exports exceed row caps, export job metadata can include continuation tokens and the manifest can include truncation.evaluations.continuations so clients can resume the same chatbook export.
• v1.1 export currently uses the shared format helpers for manifest metadata, file_inventory hashing, preview reporting, and import validation. Content types keep their v1-compatible payload layout until producer-specific v1.1 envelopes are added.

Response (Asynchronous):

{
  "success": true,
  "message": "Export job started",
  "job_id": "0c9d9a3a-6d1c-4c8f-9c84-9a0c2c2d8f77"
}

1A. Preview Full Account Export Scope

Endpoint: GET /api/v1/chatbooks/export/scope

Description: Return the redacted account-data inventory summary used by Backup all. Use this endpoint to show users what a full user-account export will cover before creating the export.

Response:

{
  "mode": "full_account",
  "total_items": 42,
  "pointer_only_count": 3,
  "sensitive_category_count": 2,
  "warning_count": 1,
  "estimated_size_bytes": 10485760,
  "categories": [
    {
      "category": "media_stored_artifacts",
      "label": "Media stored artifacts",
      "count": 5,
      "restore_status": "restorable",
      "sensitivity": "personal",
      "warning": null
    },
    {
      "category": "media_pointers",
      "label": "Media pointers",
      "count": 3,
      "restore_status": "pointer_only",
      "sensitivity": "personal",
      "warning": "External source bytes are not stored by tldw."
    }
  ]
}

The response contains labels and counts only. Sensitive values and secrets are never returned by this preview endpoint.

2. Import Chatbook

Endpoint: POST /api/v1/chatbooks/import

Description: Import content from an uploaded chatbook archive, OpenWebUI chat export JSON, or OpenWebUI webui.db database.

Request: Multipart form data

• file (form field): The import file (required)
• source_format (form field): chatbook (default), openwebui_json, or openwebui_db
• selected_openwebui_user_id (form field): required when source_format=openwebui_db; use a source user id returned by preview
• conflict_resolution (form field): skip (default) or rename for current import flows
• prefix_imported (form field): boolean, default false
• content_selections (form field): optional JSON object encoded as a string; unsupported content types are rejected
• import_media and import_embeddings (form fields): optional. For source_format=chatbook, omit them to restore media data and embeddings present in the archive. For OpenWebUI imports, omit them or keep them false; true values are rejected because OpenWebUI attachment copying uses the hydration endpoints instead.

Supported multipart fields:

{
  "source_format": "chatbook",
  "selected_openwebui_user_id": "user_abc123",
  "content_selections": "{\"conversation\":[\"conv_123\"],\"note\":[]}",
  "content_selections_json_shape": {
    "conversation": ["conv_123"],
    "note": []
  },
  "conflict_resolution": "skip",
  "prefix_imported": false,
  "import_media": null,
  "import_embeddings": null,
  "async_mode": false
}

For source_format=chatbook, the upload must be a .zip or .chatbook archive. For source_format=openwebui_json, the upload must be a safe .json filename and the server does not run ZIP validation or Chatbook manifest extraction. For source_format=openwebui_db, the upload must be a safe .db or .sqlite filename and a valid SQLite database.

OpenWebUI JSON import supports normal OpenWebUI "Export Chats" JSON files. It imports every valid chat as one tldw conversation and preserves all valid message branches through parent_message_id.

OpenWebUI webui.db database import supports uploaded SQLite databases copied from an OpenWebUI instance. Preview first, choose exactly one selected_openwebui_user_id, then import with source_format=openwebui_db. The import reads only chats owned by the selected source user. Source folders are mirrored into tldw folders under OpenWebUI / <selected user> / ..., and folder links are attached to imported conversations. Duplicate detection uses source=openwebui and a deterministic external reference for both JSON and DB imports. skip is the default duplicate behavior; rename creates an intentional second copy with a unique imported title and external reference. OpenWebUI v1 does not support overwrite, merge, admin export import, live OpenWebUI server import, embeddings import, content selections, or unreferenced attachment import. Files, images, and artifacts import as metadata references first; use the OpenWebUI attachment hydration endpoints after import to copy referenced images and register supported files from a server-local OpenWebUI data root.

For Chatbook archives, import restores all restorable account data present by default, including media records, transcripts, chunks, derived metadata, bundled stored media artifacts, embeddings, prompts, evaluations, generated documents, and supported content relationships. Pointer-only sources are restored as metadata only and reported in warnings.

Response:

{
  "source_format": "chatbook",
  "success": true,
  "message": "Chatbook imported successfully",
  "imported_items": {
    "conversation": 5,
    "note": 10,
    "character": 2
  }
}

OpenWebUI JSON Response:

{
  "source_format": "openwebui_json",
  "success": true,
  "message": "OpenWebUI chats imported successfully",
  "openwebui_result": {
    "imported_chats": 4,
    "skipped_chats": 1,
    "failed_chats": 0,
    "imported_messages": 128,
    "skipped_messages": 0,
    "duplicate_chats": 1,
    "warnings": []
  }
}

OpenWebUI Database Response:

{
  "source_format": "openwebui_db",
  "success": true,
  "message": "OpenWebUI database chats imported successfully",
  "openwebui_db_result": {
    "imported_chats": 4,
    "skipped_chats": 1,
    "failed_chats": 0,
    "imported_messages": 128,
    "duplicate_chats": 1,
    "mirrored_folders": 3,
    "folder_links": 4,
    "warnings": []
  }
}

v1.1 Import Behavior:

• v1.1 manifests must include file_inventory; an empty list is valid only when the archive has no bundled payload files.
• v1.1 archives are validated before any content writes.
• file_inventory checksum failures, missing listed files, unsafe inventory paths, and missing inventory coverage for import payloads fail the import before writes.
• Conversation image attachments referenced by bundled conversation payloads must also be covered by verified inventory entries before import proceeds.
• Unknown feature behavior follows manifest.compatibility.unsupported_feature_behavior: reject_import fails before writes; warn_and_skip and warn_lossy_import continue where safe and surface warnings in the sync import response or job warnings.

3. Preview Chatbook

Endpoint: POST /api/v1/chatbooks/preview

Description: Preview chatbook archive, OpenWebUI JSON contents, or OpenWebUI database users without importing.

Request: Multipart form data

• file: The file to preview
• source_format: chatbook (default), openwebui_json, or openwebui_db

Response:

{
  "source_format": "chatbook",
  "manifest": {
    "version": "1.0.0",
    "name": "My Chatbook",
    "description": "Research collection",
    "author": "Jane Doe",
    "created_at": "2024-01-15T10:30:00Z",
    "total_conversations": 10,
    "total_notes": 25,
    "total_characters": 3,
    "total_size_bytes": 5242880,
    "content_items": []  // Preview omits detailed items for performance
  }
}

OpenWebUI JSON Preview Response:

{
  "source_format": "openwebui_json",
  "openwebui_preview": {
    "chat_count": 5,
    "message_count": 132,
    "branched_chat_count": 2,
    "duplicate_chat_count": 1,
    "attachment_reference_count": 3,
    "malformed_chat_count": 0,
    "warnings": [],
    "items": [
      {
        "title": "Research thread",
        "external_ref": "chat_abc123",
        "message_count": 31,
        "branched": true,
        "duplicate": false,
        "warning_count": 0
      }
    ]
  }
}

OpenWebUI Database Preview Response:

{
  "source_format": "openwebui_db",
  "openwebui_db_preview": {
    "user_count": 2,
    "users": [
      {
        "source_user_id": "user_abc123",
        "display_label": "Alice",
        "email": "alice@example.test",
        "chat_count": 12,
        "message_count": 344,
        "folder_count": 5,
        "branched_chat_count": 3,
        "duplicate_chat_count": 1,
        "archived_chat_count": 2,
        "pinned_chat_count": 4,
        "attachment_reference_count": 8,
        "warning_count": 0,
        "warnings": []
      }
    ],
    "warnings": []
  }
}

For v1.1 archives the preview response may also include a deterministic preview report. These fields are optional so v1.0 preview clients can continue reading the existing manifest shape:

{
  "manifest": {
    "version": "1.1.0",
    "name": "My Chatbook",
    "content_items": []
  },
  "compatibility": {
    "status": "compatible",
    "reader_version": "1.1.0",
    "manifest_version": "1.1.0"
  },
  "features": {
    "supported": ["content_envelopes", "file_inventory"],
    "unsupported": []
  },
  "integrity": {
    "verified_files": 3,
    "failed_files": []
  },
  "lossiness": {
    "lossless": 1
  },
  "source_refs": {
    "external": 2
  },
  "warnings": [],
  "errors": []
}

compatibility reports reader and manifest compatibility. features splits manifest feature tokens into supported and unsupported sets. integrity summarizes file inventory verification. lossiness counts content-envelope lossiness modes, and source_refs counts source reference resolution statuses. warnings and errors are report-level messages; integrity errors in preview become blocking validation errors during v1.1 import.

3A. List OpenWebUI Import Scopes

Endpoint: GET /api/v1/chatbooks/openwebui/import-scopes

Description: List imported OpenWebUI scopes for the current user. The WebUI uses this after OpenWebUI JSON or database import so attachment hydration can target the last or selected import without asking users to copy tldw conversation IDs.

Response:

{
  "scopes": [
    {
      "scope_id": "openwebui-json-20260709T181500Z",
      "source_format": "openwebui_json",
      "source_user_id": "user_abc123",
      "source_user_label": "Alice",
      "conversation_count": 4,
      "attachment_reference_count": 8,
      "created_at": "2026-07-09T18:15:00Z"
    }
  ]
}

Scope summaries are user-safe and do not expose server-local source paths.

4. Preview OpenWebUI Attachment Hydration

Endpoint: POST /api/v1/chatbooks/openwebui/hydration/preview

Description: Preview hydration for files referenced by imported OpenWebUI conversations. This endpoint validates the server-local OpenWebUI data root, scans the selected imported tldw conversations for preserved OpenWebUI attachment metadata, resolves referenced files through webui.db and uploads/, and returns counts and warnings without copying files.

Hydration v1 is reference-driven. It does not scan every file under uploads/ and does not connect to a live OpenWebUI server.

Auth and path requirements:

• Single-user mode is allowed.
• Multi-user mode requires an owner/admin principal authorized for the imported conversations.
• openwebui_data_root must be a server-local path containing webui.db and uploads/.
• The data root and resolved files must be under Files.ingestion_source_allowed_roots, INGESTION_SOURCE_ALLOWED_ROOTS, or TLDW_INGESTION_SOURCE_ALLOWED_ROOTS.

Request: JSON

{
  "openwebui_data_root": "/srv/openwebui/data",
  "scope": {
    "import_scope_id": "openwebui-json-20260709T181500Z"
  },
  "process_supported_files": false
}

Fields:

• openwebui_data_root: required server-local OpenWebUI data root.
• scope.import_scope_id: selected OpenWebUI import scope to hydrate. Prefer this for normal WebUI flows.
• scope.conversation_ids: imported tldw conversation IDs to scan. Use as an advanced manual override. Empty means no conversations are selected unless import_scope_id is provided.
• scope.source_user_id: optional OpenWebUI user id used for database chat_file fallback lookups.
• process_supported_files: optional, defaults to false. When true, preview includes supported-file processing intent for non-image files.

Response:

{
  "scope": {
    "import_scope_id": "openwebui-json-20260709T181500Z",
    "conversation_ids": ["conv_123", "conv_456"],
    "source_user_id": "user_abc123"
  },
  "process_supported_files": false,
  "summary": {
    "referenced_files": 3,
    "returned_items": 3,
    "omitted_items": 0,
    "resolved_files": 2,
    "image_files": 1,
    "media_files": 1,
    "missing_files": 1,
    "unsupported_files": 0,
    "failed_files": 0,
    "hydrated_images": 0,
    "registered_media_files": 0,
    "already_hydrated": 0,
    "processed_files": 0,
    "warning_count": 1
  },
  "items": [
    {
      "conversation_id": "conv_123",
      "message_id": "msg_123",
      "file_id": "file_abc",
      "status": "preview_ready",
      "file_kind": "image",
      "mime_type": "image/png"
    }
  ],
  "warnings": ["1 referenced file was missing"]
}

items is capped to protect API clients from very large responses. Use summary.referenced_files, summary.returned_items, and summary.omitted_items to detect whether the response was truncated. Common warnings include missing_file, unsupported_file_type, file_too_large, path_rejected, and paths outside allowed roots.

5. Create OpenWebUI Attachment Hydration Job

Endpoint: POST /api/v1/chatbooks/openwebui/hydration/jobs

Description: Enqueue the same hydration operation as a background job. Use preview first; the WebUI gates job creation on a successful preview for the same payload.

Image files are copied into tldw-owned storage and message metadata is updated. Non-image files are registered in the Media DB when available. File processing remains opt-in through process_supported_files=true; with the default false, supported non-image files are registered but not processed.

Hydration output storage uses OPENWEBUI_HYDRATION_MEDIA_STORAGE_PATH when set, otherwise MEDIA_STORAGE_PATH, otherwise the server's default media storage directory.

Request: Same JSON shape as preview.

Response:

{
  "job_id": "123",
  "job_uuid": "7d0d6a9a-4a48-4d35-bd4e-3e33b6f2dfb1",
  "status": "pending",
  "domain": "chatbooks",
  "queue": "default",
  "job_type": "openwebui_attachment_hydration",
  "owner_user_id": "1",
  "created_at": "2026-05-11T17:00:00Z",
  "updated_at": "2026-05-11T17:00:00Z",
  "result": null,
  "error": null
}

6. Get OpenWebUI Attachment Hydration Job

Endpoint: GET /api/v1/chatbooks/openwebui/hydration/jobs/{job_id}

Description: Return one OpenWebUI hydration job visible to the current caller. Non-admin users can only read their own jobs.

Response:

{
  "job_id": "123",
  "status": "completed",
  "job_type": "openwebui_attachment_hydration",
  "result": {
    "summary": {
      "referenced_files": 3,
      "returned_items": 3,
      "omitted_items": 0,
      "resolved_files": 2,
      "hydrated_images": 1,
      "registered_media_files": 1,
      "processed_files": 0,
      "warning_count": 1
    },
    "warnings": ["1 referenced file was missing"]
  },
  "error": null
}

7. Download Chatbook

Endpoint: GET /api/v1/chatbooks/download/{job_id}

Description: Download a completed export.

Response: Binary file stream (application/zip)

Signed URLs (optional):

• If CHATBOOKS_SIGNED_URLS=true and CHATBOOKS_SIGNING_SECRET is set, the download link is signed and requires query params exp (Unix timestamp) and token (HMAC SHA256 of "{job_id}:{exp}").
• If CHATBOOKS_ENFORCE_EXPIRY=true, the server enforces job-level expires_at and returns 410 when expired.
• Invalid or missing signature returns 403; an expired exp parameter returns 410.

8. List Export Jobs

Endpoint: GET /api/v1/chatbooks/export/jobs

Query Parameters:

• limit: Maximum results (1-1000, default: 100)
• offset: Skip results (default: 0)

Response:

{
  "jobs": [
    {
      "job_id": "0c9d9a3a-6d1c-4c8f-9c84-9a0c2c2d8f77",
      "status": "completed",
      "chatbook_name": "Weekly Backup",
      "created_at": "2024-01-15T10:00:00Z",
      "completed_at": "2024-01-15T10:05:00Z",
      "progress_percentage": 100,
      "total_items": 50,
      "processed_items": 50,
      "file_size_bytes": 10485760,
  "download_url": "/api/v1/chatbooks/download/0c9d9a3a-6d1c-4c8f-9c84-9a0c2c2d8f77",
      "expires_at": "2024-02-14T10:05:00Z"
    }
  ],
  "total": 15
}

9. Get Export Job Status

Endpoint: GET /api/v1/chatbooks/export/jobs/{job_id}

Response: Same as individual job in list response.

10. List Import Jobs

Endpoint: GET /api/v1/chatbooks/import/jobs

Query Parameters:

• limit: Maximum results (1-1000, default: 100)
• offset: Skip results (default: 0)

Response:

{
  "jobs": [
    {
      "job_id": "cb_import_20240115_def456",
      "status": "completed",
      "chatbook_path": "/tmp/uploads/chatbook.zip",
      "created_at": "2024-01-15T11:00:00Z",
      "completed_at": "2024-01-15T11:03:00Z",
      "progress_percentage": 100,
      "total_items": 45,
      "processed_items": 45,
      "successful_items": 43,
      "failed_items": 0,
      "skipped_items": 2,
      "conflicts": [],
      "warnings": []
    }
  ],
  "total": 8
}

11. Get Import Job Status

Endpoint: GET /api/v1/chatbooks/import/jobs/{job_id}

Response: Same as individual job in list response.

12. Cancel Export Job

Endpoint: DELETE /api/v1/chatbooks/export/jobs/{job_id}

Response:

{
  "message": "Export job 0c9d9a3a-6d1c-4c8f-9c84-9a0c2c2d8f77 cancelled"
}

13. Cancel Import Job

Endpoint: DELETE /api/v1/chatbooks/import/jobs/{job_id}

Response:

{
  "message": "Import job cb_import_20240115_def456 cancelled"
}

14. Remove Export Job

Endpoint: DELETE /api/v1/chatbooks/export/jobs/{job_id}/remove

Removes a completed or cancelled export job (and deletes the exported archive).

Response:

{
  "success": true,
  "message": "Export job 0c9d9a3a-6d1c-4c8f-9c84-9a0c2c2d8f77 removed",
  "job_id": "0c9d9a3a-6d1c-4c8f-9c84-9a0c2c2d8f77"
}

15. Remove Import Job

Endpoint: DELETE /api/v1/chatbooks/import/jobs/{job_id}/remove

Removes a completed or cancelled import job.

Response:

{
  "success": true,
  "message": "Import job cb_import_20240115_def456 removed",
  "job_id": "cb_import_20240115_def456"
}

16. Cleanup Expired Exports

Endpoint: POST /api/v1/chatbooks/cleanup

Description: Remove expired export files to free storage. Scheduled cleanup runs via the Chatbooks worker; use this endpoint to trigger a manual sweep.

Response:

{
  "deleted_count": 5
}

17. Service Health

Lightweight liveness check for the Chatbooks subsystem.

Endpoint: GET /api/v1/chatbooks/health

Response:

{
  "service": "chatbooks",
  "status": "healthy|degraded|unhealthy",
  "timestamp": "2024-01-15T10:30:00Z",
  "components": {
    "storage_base": {"path": "<USER_DB_BASE_DIR>", "exists": true, "writable": true}
  }
}

USER_DB_BASE_DIR is defined in tldw_Server_API.app.core.config (defaults to Databases/user_databases/ under the project root). Override via environment variable or Config_Files/config.txt as needed.

Data Models

ContentType Enum

- conversation
- note
- character
- media
- embedding
- prompt
- evaluation
- world_book
- dictionary
- generated_document

ConflictResolution Enum

- skip: Skip items that already exist
- overwrite: Replace existing items
- rename: Add with modified name
- merge: Combine with existing (future feature)

OpenWebUI JSON and database imports support skip and rename in v1.

ExportStatus Enum

- pending: Job queued
- in_progress: Currently processing
- completed: Successfully finished
- failed: Error occurred
- cancelled: Manually stopped
- expired: File removed

ImportStatus Enum

- pending: Job queued
- validating: Checking file integrity
- in_progress: Importing content
- completed: Successfully finished
- failed: Error occurred
- cancelled: Manually stopped

Manifest Metadata (selected fields)

• metadata.binary_limits: Per content-type max bundled bytes applied during export.
• truncation.evaluations.continuations: Continuation tokens for resumable evaluation exports.
• account_inventory: v1.1 list of account inventory categories, restore status, sensitivity class, and user-facing warning text.
• account_inventory_summary: v1.1 redacted summary containing category counts, pointer-only counts, warning counts, archive size, and post_write_verification.
• v1.1 manifests may add features_used, producer, source_instance, compatibility, and file_inventory. These are produced only for opt-in v1.1 exports.

Sensitive values may be present in archive payload files only when required for restore. They are redacted from preview responses, manifest summaries, job metadata, warnings, logs, and audit-facing metadata.

Error Handling

Error Response Format

{
  "detail": "Detailed error message",
  "error_type": "ValidationError",
  "job_id": "0c9d9a3a-6d1c-4c8f-9c84-9a0c2c2d8f77",
  "suggestions": [
    "Check file format",
    "Ensure file size is under 100MB"
  ]
}

Common Error Types

• ValidationError: Invalid input parameters
• AuthenticationError: Missing or invalid token
• AuthorizationError: Insufficient permissions
• NotFoundError: Resource doesn't exist
• ConflictError: Operation conflicts with current state
• QuotaExceededError: User quota limit reached
• FileTooLargeError: File exceeds size limit
• RateLimitError: Too many requests

HTTP Status Codes

• 200: Success
• 202: Accepted (async operation started)
• 400: Bad Request
• 401: Unauthorized
• 403: Forbidden
• 404: Not Found
• 409: Conflict
• 413: Payload Too Large
• 429: Too Many Requests
• 500: Internal Server Error

Rate Limiting

Per-user limits enforced at the endpoint level:

• Export: 5/minute
• Import: 5/minute
• Preview: 10/minute
• Download: 20/minute

List/status endpoints may be subject to global API rate limits but have no dedicated per-route limiter. Exceeded limits return HTTP 429.

Code Examples

Python Example

import json
import time
import uuid
from urllib.request import Request, urlopen

# Configuration
API_BASE = "http://localhost:8000/api/v1"
TOKEN = "your-jwt-token"
headers = {"Authorization": f"Bearer {TOKEN}"}  # In single-user mode use: {"X-API-KEY": API_KEY}

def request_json(method, url, payload=None, headers=None):
    data = json.dumps(payload).encode("utf-8") if payload is not None else None
    hdrs = {"Content-Type": "application/json"}
    if headers:
        hdrs.update(headers)
    req = Request(url, data=data, headers=hdrs, method=method)
    with urlopen(req) as resp:
        return json.loads(resp.read().decode("utf-8"))

def download_file(url, headers, dest_path, chunk_size=8192):
    req = Request(url, headers=headers, method="GET")
    with urlopen(req) as resp, open(dest_path, "wb") as f:
        while True:
            chunk = resp.read(chunk_size)
            if not chunk:
                break
            f.write(chunk)

def encode_multipart(fields, files):
    boundary = uuid.uuid4().hex
    body = bytearray()

    def add_line(line):
        body.extend(line.encode("utf-8"))
        body.extend(b"\r\n")

    for name, value in fields.items():
        add_line(f"--{boundary}")
        add_line(f'Content-Disposition: form-data; name="{name}"')
        add_line("")
        add_line(str(value))

    for name, filename, content, content_type in files:
        add_line(f"--{boundary}")
        add_line(f'Content-Disposition: form-data; name="{name}"; filename="{filename}"')
        add_line(f"Content-Type: {content_type or 'application/octet-stream'}")
        add_line("")
        body.extend(content)
        body.extend(b"\r\n")

    add_line(f"--{boundary}--")
    return boundary, bytes(body)

# Create a chatbook export (async)
data = request_json(
    "POST",
    f"{API_BASE}/chatbooks/export",
    {
        "name": "Weekly Backup",
        "description": "Complete backup of all content",
        "include_media": True,
        "include_embeddings": True,
        "async_mode": True,
    },
    headers=headers,
)
job_id = data.get("job_id")
if job_id:
    print(f"Export started: {job_id}")

    # Monitor export job
    while True:
        status_data = request_json(
            "GET",
            f"{API_BASE}/chatbooks/export/jobs/{job_id}",
            headers=headers,
        )

        print(f"Progress: {status_data['progress_percentage']}%")

        if status_data["status"] == "completed":
            # Download the chatbook
            download_file(
                f"{API_BASE}/chatbooks/download/{job_id}",
                headers=headers,
                dest_path="my_backup.chatbook",
            )
            print("Download complete!")
            break
        elif status_data["status"] == "failed":
            print(f"Export failed: {status_data['error_message']}")
            break

        time.sleep(5)  # Check every 5 seconds

# Import a chatbook (options as multipart form fields)
with open("my_backup.chatbook", "rb") as f:
    boundary, body = encode_multipart(
        {
            "source_format": "chatbook",
            "conflict_resolution": "skip",
            "prefix_imported": "false",
        },
        [("file", "my_backup.chatbook", f.read(), "application/octet-stream")],
    )
    upload_headers = {
        **headers,
        "Content-Type": f"multipart/form-data; boundary={boundary}",
    }
    req = Request(
        f"{API_BASE}/chatbooks/import",
        data=body,
        headers=upload_headers,
        method="POST",
    )
    with urlopen(req) as resp:
        if resp.status != 200:
            print(f"Import failed with status {resp.status}")
        else:
            result = json.loads(resp.read().decode("utf-8"))
            print(f"Imported: {result['imported_items']}")

JavaScript Example

const API_BASE = 'http://localhost:8000/api/v1';
const TOKEN = 'your-jwt-token';

// Create chatbook
async function createChatbook() {
  const response = await fetch(`${API_BASE}/chatbooks/export`, {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${TOKEN}`,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify({
      name: 'Weekly Backup',
      description: 'Complete backup',
      include_media: true,
      include_embeddings: true,
      async_mode: true
    })
  });

  const data = await response.json();
  return data.job_id;
}

// Monitor job progress
async function monitorJob(jobId) {
  while (true) {
    const response = await fetch(
      `${API_BASE}/chatbooks/export/jobs/${jobId}`,
      {
        headers: {
          'Authorization': `Bearer ${TOKEN}`
        }
      }
    );

    const status = await response.json();
    console.log(`Progress: ${status.progress_percentage}%`);

    if (status.status === 'completed') {
      return status.download_url;
    } else if (status.status === 'failed') {
      throw new Error(status.error_message);
    }

    // Wait 5 seconds before checking again
    await new Promise(resolve => setTimeout(resolve, 5000));
  }
}

// Import chatbook (options as multipart form fields)
async function importChatbook(file) {
  const formData = new FormData();
  formData.append('file', file);
  formData.append('source_format', 'chatbook');
  formData.append('conflict_resolution', 'skip');
  formData.append('prefix_imported', 'false');

  const response = await fetch(`${API_BASE}/chatbooks/import`, {
    method: 'POST',
    headers: {
      'Authorization': `Bearer ${TOKEN}`
    },
    body: formData
  });

  return await response.json();
}

// Usage
createChatbook()
  .then(jobId => monitorJob(jobId))
  .then(downloadUrl => {
    console.log(`Download at: ${downloadUrl}`);
  })
  .catch(error => {
    console.error('Export failed:', error);
  });

Migration Guide

Migrating from Single-User to Multi-User

When upgrading from the single-user version to multi-user with authentication:

1. Export your data before migration:

   curl -X POST "http://localhost:8000/api/v1/chatbooks/export" \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer <your-jwt-token>" \
     -d '{"name": "Pre-migration backup", "description": "Complete backup before auth migration", "include_media": true, "include_embeddings": true}'
   

   Compatibility note: omitting content_selections and sending "content_selections": {} both mean full user-account export. Only a non-empty content_selections object starts explicit allowlist mode.

2. Enable authentication in the new version

3. Import your data with your new user token:

   curl -X POST "http://localhost:8000/api/v1/chatbooks/import" \
     -H "Authorization: Bearer <new-token>" \
     -F "file=@backup.chatbook"
   

Best Practices

1. Regular Backups: Schedule weekly or monthly exports
2. Selective Exports: Export only active projects to reduce file size when you intentionally do not need a full backup
3. Conflict Strategy: Use "skip" for regular backups, "rename" for shared imports
4. Media Management: Full backups include restorable media data and stored account artifacts; use selective export for text-only archives
5. Job Monitoring: Always check job status for async operations
6. Error Recovery: Keep original files until import is confirmed successful

Quota Limits

User Tier	Daily Exports	Daily Imports	Max File Size	Retention
Free	5	5	100 MB	7 days
Basic	20	20	500 MB	30 days
Premium	100	100	2 GB	90 days
Enterprise	Unlimited	Unlimited	10 GB	365 days

Security Considerations

1. File Validation: All uploads are validated for format and content
2. Path Traversal: File paths are sanitized to prevent directory traversal
3. Size Limits: Enforced to prevent resource exhaustion
4. User Isolation: Content is strictly isolated between users
5. Temporary Files: Cleaned up automatically after processing
6. Encryption: Consider encrypting chatbooks before sharing

---

Last updated: 2025-12-29 API Version: 1.0.0

Configuration

Environment variables controlling Chatbooks job backends and downloads:

• CHATBOOKS_JOBS_BACKEND: Core-only; overrides are ignored (kept for compatibility).
• CHATBOOKS_CORE_WORKER_ENABLED: true|false to start the Chatbooks worker.
• Deprecated: TLDW_USE_PROMPT_STUDIO_QUEUE (ignored; legacy backend removed).

Signed downloads:

• CHATBOOKS_SIGNED_URLS=true|false - enable HMAC-signed download URLs.
• CHATBOOKS_SIGNING_SECRET - shared secret for signing.
• CHATBOOKS_ENFORCE_EXPIRY=true|false - enforce job expires_at.
• CHATBOOKS_URL_TTL_SECONDS - default expiry TTL for generated links.

Retention and cleanup:

• CHATBOOKS_EXPORT_RETENTION_DEFAULT_HOURS - retention window for completed exports before expiry (default 24).
• CHATBOOKS_CLEANUP_INTERVAL_SEC - scheduled cleanup cadence in seconds (set to 0 to disable scheduling).

Export limits:

• CHATBOOKS_EVAL_EXPORT_MAX_ROWS - max rows exported per evaluation run (default 200).
• CHATBOOKS_BINARY_LIMITS_MB - JSON map of content type to max bundled size in MB (for example, {"media": 0, "conversations": 10, "generated_docs": 25}).

Template/import controls:

• CHATBOOKS_TEMPLATE_MODE - default manifest template mode for exports (pass_through|render_on_export, default pass_through).
• CHATBOOKS_TEMPLATE_DEFAULTS_JSON - optional JSON object merged into template defaults.
• CHATBOOKS_TEMPLATE_TIMEZONE - default timezone used for template rendering (default UTC).
• CHATBOOKS_TEMPLATE_LOCALE - optional default locale for template rendering.
• CHATBOOKS_IMPORT_DICT_STRICT - when true, Chatbooks import skips embedded dictionaries that contain fatal validation errors and continues best-effort for the rest.