Lotti Architecture

September 10, 2026 · View on GitHub

Table of Contents

Overview

Lotti is a privacy-first personal assistant built with Flutter, featuring local-first data storage, AI integration, and end-to-end encrypted synchronization. The architecture prioritizes data ownership, privacy, and extensibility while providing powerful AI capabilities.

Core Principles

  1. Local-First: All data is stored locally using SQLite, with no cloud dependency
  2. Privacy by Design: User content leaves a device only for AI inference you configured or user-enabled end-to-end encrypted Matrix sync, both of which can then run automatically. Sync homeservers receive, store and relay ciphertext plus the associated account and traffic metadata; see PRIVACY.md for the precise boundaries
  3. Modular Architecture: Features are organized as independent modules with clear boundaries
  4. Provider Agnostic: AI capabilities work with multiple providers (OpenAI, Anthropic, Gemini, Ollama)
  5. Cross-Platform: Single codebase for iOS, macOS, Android, Windows, and Linux

High-Level Architecture

┌─────────────────────────────────────────────────────────────┐
│                        UI Layer (Flutter)                    │
├─────────────────────────────────────────────────────────────┤
│                     Feature Modules                          │
│  ┌──────────┬────────────┬──────────┬──────────────────┐   │
│  │ Tasks    │ Agents     │ Journal  │ Habits & Health  │   │
│  ├──────────┼────────────┼──────────┼──────────────────┤   │
│  │ Audio    │ Categories │ Sync     │ Settings         │   │
│  └──────────┴────────────┴──────────┴──────────────────┘   │
├─────────────────────────────────────────────────────────────┤
│                     Core Services                            │
│  ┌──────────────┬───────────────┬──────────────────────┐   │
│  │ Database     │ AI Providers  │ Audio Processing     │   │
│  │ (SQLite)     │ Integration   │ (Whisper)           │   │
│  └──────────────┴───────────────┴──────────────────────┘   │
├─────────────────────────────────────────────────────────────┤
│                  Infrastructure Layer                        │
│  ┌──────────────┬───────────────┬──────────────────────┐   │
│  │ Persistence  │ Encryption    │ Background Tasks     │   │
│  │ & Migration  │ (Matrix Sync) │ & Notifications      │   │
│  └──────────────┴───────────────┴──────────────────────┘   │
└─────────────────────────────────────────────────────────────┘

Feature Modules

AI Integration

  • AI Module: Core AI provider abstraction and configuration
  • Agents: Scoped assistants, improvement flows, and shared text/voice conversation UI

Content Management

  • Tasks: Task lifecycle management with AI-enhanced summaries
  • Journal: Text and audio journal entries with transcription
  • Categories: Organize content with configurable AI providers per category

Audio & Speech

  • Speech: Audio recording and transcription using Whisper or cloud providers

Data & Visualization

  • Calendar: Time-based views of entries and activities
  • Dashboards: Analytics and insights from collected data
  • Habits: Habit tracking and completion monitoring
  • Surveys: Custom questionnaires and assessments

System Features

  • Settings: Configuration management and preferences
  • Sync: End-to-end encrypted synchronization via Matrix (requires Matrix account - self-hosted or public homeserver)
  • Tags: Flexible tagging system for organization
  • User Activity: Activity tracking and analytics

Data Flow

1. Local Data Storage

User Input → SQLite Database → UI Updates

All user data is stored in a local SQLite database with support for:

  • Full-text search
  • Efficient querying
  • Data export/import
  • Incremental backups

2. AI Processing Pipeline

User Request → Category Config → Provider Selection → API Call → Response Processing
  • Users can configure different AI providers per category
  • Requests are routed to the appropriate provider
  • Responses are processed and stored locally
  • No data retention on provider side (with appropriate plans)

3. Synchronization

Local Changes → Encryption → Matrix Protocol → Other Devices
  • Changes are encrypted locally
  • Transmitted via Matrix's decentralized network
  • Decrypted on receiving devices
  • Conflict resolution handled automatically

AI Provider Architecture

Provider Abstraction

The system supports multiple AI providers through a unified interface.

Note: The interface below is conceptual to illustrate the design. The production code uses configuration models such as AiConfigInferenceProvider and repositories (for example lib/features/ai/conversation/conversation_repository.dart) to orchestrate requests and streaming.

abstract class AiProvider {
  Stream<String> sendMessage(String message, List<Message> context);
  Future<String> transcribe(AudioData audio);
  bool get supportsStreaming;
  bool get supportsAudio;
}

Error handling and retries

  • Provider‑specific error parsing (e.g., model not found, rate limits) with user‑friendly messages
  • Exponential backoff for transient errors; fail fast for configuration issues
  • Optional fallbacks between configured providers when applicable

Supported Providers

  1. OpenAI: GPT-4, GPT-3.5, Whisper
  2. Anthropic: Claude 3.5, Claude 3
  3. Google: Gemini Pro, Gemini Flash
  4. Ollama: Local models (Llama, Mistral, etc.)
  5. OpenAI-Compatible: Any provider with compatible API

Configuration Management

  • Per-category provider selection
  • API key management (stored securely)
  • Model selection within providers
  • Fallback strategies for failures

Security Architecture

Data Protection

  • At Rest (today): Local databases are stored as plain SQLite. We recommend enabling full‑disk/device encryption (e.g., FileVault on macOS, BitLocker on Windows, File‑based encryption on Android, LUKS on Linux)
  • In Transit: TLS for all network communication
  • Sync: End-to-end encryption via Matrix
  • AI Calls: HTTPS with API key authentication

Key Storage (API keys and secrets)

Lotti uses OS‑backed secure storage via flutter_secure_storage:

  • iOS: Keychain Services
  • macOS: Keychain Services
  • Android: Android Keystore (keys) with encrypted SharedPreferences
  • Windows: Credential Locker (DPAPI)
  • Linux: Secret Service (libsecret; via GNOME Keyring/KWallet depending on environment)

Secrets stored in secure storage include AI provider API keys and tokens, and sync credentials (e.g., Matrix access tokens).

Data at Rest (Databases)

  • Current state: SQLite databases (Drift) are stored unencrypted
  • Recommended mitigation: rely on OS/device full‑disk encryption and user account protections
  • Roadmap: optional database‑level encryption for SQLite (SQLCipher)

Backups

  • Secret backup/restore is delegated to the OS keystore.
  • On iOS and macOS, Lotti stores secrets non‑synchronizable: SecureStorage (lib/features/sync/secure_storage.dart) passes only accountName to IOSOptions/MacOsOptions, and flutter_secure_storage defaults synchronizable to false. API keys and Matrix credentials therefore stay on the device and are not copied to iCloud Keychain. Enabling iCloud sync for them would require setting synchronizable: true explicitly.
  • Consequence: reinstalling or moving to a new device does not carry secrets across — they are re-entered, or arrive through device pairing.

References (platform APIs/libraries)

  • Flutter secure storage: https://pub.dev/packages/flutter_secure_storage
  • Apple Keychain Services (iOS/macOS)
  • Android Keystore System + EncryptedSharedPreferences
  • Windows Data Protection API (Credential Locker)
  • Linux Secret Service (libsecret)

Privacy Controls

  • No telemetry or analytics
  • No vendor/cloud accounts required for core functionality (local-only use)
  • Multi-device sync requires a Matrix account (self-hosted or public homeserver) - no vendor lock-in as Matrix is decentralized
  • AI processing is opt-in per category; enabling it is the consent, after which agents and transcription can call the configured provider without a further prompt
  • Neither content nor secrets leave the device except to destinations the user configured — the sync homeserver and the AI providers they chose. Secrets are stored non‑synchronizable, so they are not carried out by OS keychain sync either (see Backups)

Testing Strategy

Unit Tests

  • Core business logic
  • Data models and serialization
  • Service layer functionality
  • See repository Codecov badge for current coverage

Integration Tests

  • Database operations
  • AI provider communication
  • Sync functionality
  • Platform-specific features

Widget Tests

  • UI component behavior
  • User interaction flows
  • State management
  • Accessibility compliance

Build & Deployment

Code Generation

make build_runner  # Generate code for serialization, routing, etc.

Platform Builds

  • iOS/macOS: Xcode, TestFlight distribution
  • Android: Gradle, APK/AAB generation
  • Windows: MSIX packaging
  • Linux: Flatpak, AppImage, tar.gz

Continuous Integration

  • GitHub Actions for all platforms
  • Automated testing on PR
  • Release builds on tags
  • TestFlight deployment for Apple platforms

Performance Considerations

Optimization Strategies

  1. Lazy Loading: Load data on demand
  2. Pagination: Handle large datasets efficiently
  3. Caching: In-memory caches for frequently accessed data
  4. Background Processing: Heavy operations off main thread
  5. Streaming: Real-time AI responses without blocking

Resource Management

  • Efficient memory usage
  • Battery-conscious background tasks
  • Network request batching
  • Audio file compression

Future Architecture Plans

Planned Enhancements

  1. Plugin System: Extensible architecture for custom features
  2. Local AI Models: Expanded support for on-device inference
  3. Advanced Analytics: Local ML for pattern recognition
  4. Federation: Decentralized sharing with privacy
  5. Voice Interface: Hands-free interaction

Scalability Considerations

  • SQLite handles single-user data efficiently at any scale
  • Incremental sync optimization
  • Parallel AI processing
  • Multi-device coordination

Development Guidelines

Implementation Notes

  • State management and dependency injection use Riverpod
  • Persistence via Drift over SQLite; schemas live under lib/database and feature‑specific *.drift files
  • Localization with Flutter gen‑l10n; ARB files in lib/l10n/ and generation via make l10n
  • Code generation via build_runner; run make build_runner

Code Organization

lib/
├── features/       # Feature modules
├── services/       # Core services
├── models/         # Data models
├── widgets/        # Shared UI components
├── utils/          # Utility functions
└── main.dart       # Application entry

Best Practices

  1. Follow Flutter style guide
  2. Write tests for new features
  3. Document complex logic
  4. Use dependency injection
  5. Handle errors gracefully

Contributing

See CONTRIBUTING.md for guidelines on:

  • Code style and standards
  • Testing requirements
  • Pull request process
  • Community guidelines

For development environment setup, see DEVELOPMENT.md.