@capgo/capacitor-llm

Adds support for LLM locally run for Capacitor

It uses Apple Intelligence for the iOS system model, MediaPipe for existing .task models, and ExecuTorch .pte models on both iOS and Android.

Mac Catalyst: Native iOS functionality is disabled for Mac Catalyst builds. MediaPipe pods are skipped and native calls will return an unsupported response; use an iOS/iPadOS target for native features.

Documentation

The most complete doc is available here: https://capgo.app/docs/plugins/llm/

Compatibility

Note: The major version of this plugin follows the major version of Capacitor. Use the version that matches your Capacitor installation (e.g., plugin v8 for Capacitor 8). Only the latest major version is actively maintained.

Installation

bun add @capgo/capacitor-llm
bunx cap sync

iOS Additional Setup for Custom Models

Apple Intelligence works without bundled model files on supported iOS versions. For custom models, the plugin supports MediaPipe through CocoaPods and ExecuTorch when the host app links ExecuTorch with Swift Package Manager. The plugin package remains iOS 15 compatible; iOS ExecuTorch requires an iOS 17 or newer app target because ExecuTorch requires iOS 17.

Using CocoaPods: The MediaPipe dependencies are already configured in the podspec. Make sure to run pod install after adding the plugin.

ExecuTorch is not provided by the CocoaPods integration. CocoaPods installs MediaPipe only. CocoaPods builds reject engine: 'executorch' with a clear runtime error unless the app also links ExecuTorch separately.

Note about Static Framework Warning: When running pod install, you may see a warning about transitive dependencies with statically linked binaries. To fix this, update your Podfile:

# Change this:
use_frameworks!

# To this:
use_frameworks! :linkage => :static

# And add this to your post_install hook:
post_install do |installer|
  assertDeploymentTarget(installer)

  # Fix for static framework dependencies
  installer.pods_project.targets.each do |target|
    target.build_configurations.each do |config|
      config.build_settings['BUILD_LIBRARY_FOR_DISTRIBUTION'] = 'YES'
    end

    # Specifically for MediaPipe pods
    if target.name.include?('MediaPipeTasksGenAI')
      target.build_configurations.each do |config|
        config.build_settings['ENABLE_BITCODE'] = 'NO'
      end
    end
  end
end

Using Swift Package Manager: The plugin's Package.swift does not link ExecuTorch directly because that would force every Swift Package Manager user onto iOS 17 or newer, even when they only use Apple Intelligence or MediaPipe.

To use ExecuTorch on iOS, add it to your app target:

1. Set the app target deployment version to iOS 17 or newer.
2. In Xcode, open the app project, select Package Dependencies, and add https://github.com/pytorch/executorch.git.
3. Use the swiftpm-1.2.0 branch, or the ExecuTorch branch that matches the runtime version you want.
4. Add these products to the app target: executorch_llm, backend_xnnpack, kernels_llm, kernels_optimized, kernels_quantized, and kernels_torchao.
5. Add the .pte model and tokenizer file to the app target's Copy Bundle Resources, or download them at runtime.

If those ExecuTorch products are not linked into the app target, setModel({ engine: 'executorch' }) rejects with a clear runtime error. MediaPipe GenAI still does not officially support SPM, so use CocoaPods for MediaPipe .task models.

Adding a Model to Your App

The simplest cross-platform custom model path is ExecuTorch. It uses the same kind of .pte model file on iOS and Android, plus a tokenizer file.

ExecuTorch Models (iOS and Android)

iOS uses ExecuTorch only when the host app links the ExecuTorch Swift Package products. It is not included by CocoaPods or by the plugin's Package.swift, so an app without those products can use Apple Intelligence or MediaPipe but will refuse engine: 'executorch'. Android uses the ExecuTorch Maven package.

Bundle the model files:

• iOS: add the .pte model and tokenizer file to your app target's Copy Bundle Resources.
• Android: place the .pte model and tokenizer file under android/app/src/main/assets/, then reference them with /android_asset/....

import { Capacitor } from '@capacitor/core';
import { CapgoLLM } from '@capgo/capacitor-llm';

const isAndroid = Capacitor.getPlatform() === 'android';

await CapgoLLM.setModel({
  engine: 'executorch',
  path: isAndroid ? '/android_asset/model.pte' : 'model.pte',
  tokenizerPath: isAndroid ? '/android_asset/tokenizer.model' : 'tokenizer.model',
  maxTokens: 2048,
  sequenceLength: 2048,
  temperature: 0.8,
});

const { id: chatId } = await CapgoLLM.createChat();

engine: 'auto' also selects ExecuTorch when the model path ends in .pte or when tokenizerPath is provided. Passing engine: 'executorch' is recommended when loading ExecuTorch models so failures are explicit.

MediaPipe Models

MediaPipe remains available for existing .task models. Android models usually need both .task and .litertlm files. iOS MediaPipe support is available through CocoaPods and remains experimental for some .task files.

await CapgoLLM.setModel({
  path: '/android_asset/gemma-3-270m-it-int8.task',
  modelType: 'task',
  maxTokens: 2048,
  topk: 40,
  temperature: 0.8,
});

Apple Intelligence

On supported iOS devices, Apple Intelligence can be used without bundling a model.

await CapgoLLM.setModel({
  path: 'Apple Intelligence',
  engine: 'apple',
});

Downloading Models at Runtime

Use downloadModel to keep large model files out of the app bundle. For ExecuTorch, companionUrl can point at the tokenizer file.

const result = await CapgoLLM.downloadModel({
  url: 'https://your-server.com/models/model.pte',
  companionUrl: 'https://your-server.com/models/tokenizer.model',
  filename: 'model.pte',
});

await CapgoLLM.setModel({
  engine: 'executorch',
  path: result.path,
  tokenizerPath: result.companionPath,
  sequenceLength: 2048,
});

Usage Example

import { CapgoLLM } from '@capgo/capacitor-llm';

const { readiness } = await CapgoLLM.getReadiness();
console.log('LLM readiness:', readiness);

const { id: chatId } = await CapgoLLM.createChat();

CapgoLLM.addListener('textFromAi', (event) => {
  console.log('AI:', event.text);
});

CapgoLLM.addListener('aiFinished', (event) => {
  console.log('AI finished responding to chat:', event.chatId);
});

await CapgoLLM.sendMessage({
  chatId,
  message: 'Hello! How are you today?',
});

API

createChat() => Promise<{ id: string; instructions?: string; }>

sendMessage(options: { chatId: string; message: string; }) => Promise<void>

getReadiness() => Promise<{ readiness: string; }>

setModel(options: ModelOptions) => Promise<void>

downloadModel(options: DownloadModelOptions) => Promise<DownloadModelResult>

addListener(eventName: 'textFromAi', listenerFunc: (event: TextFromAiEvent) => void) => Promise<{ remove: () => Promise<void>; }>

addListener(eventName: 'aiFinished', listenerFunc: (event: AiFinishedEvent) => void) => Promise<{ remove: () => Promise<void>; }>

addListener(eventName: 'downloadProgress', listenerFunc: (event: DownloadProgressEvent) => void) => Promise<{ remove: () => Promise<void>; }>

addListener(eventName: 'readinessChange', listenerFunc: (event: ReadinessChangeEvent) => void) => Promise<{ remove: () => Promise<void>; }>

getPluginVersion() => Promise<{ version: string; }>

Example App Model Setup

The example app can use either the older MediaPipe assets or the new ExecuTorch assets.

Recommended Custom Path: ExecuTorch

1. Export or download an ExecuTorch .pte model and matching tokenizer file.
2. Add both files to the iOS app bundle, or place both files in example-app/android/app/src/main/assets/ for Android.
3. Call setModel with engine: 'executorch', path, and tokenizerPath.

Legacy MediaPipe Path

Android still supports Gemma 3 LiteRT assets from Kaggle. Download both files and place them in example-app/android/app/src/main/assets/:

• gemma-3-270m-it-int8.task
• gemma-3-270m-it-int8.litertlm

iOS MediaPipe remains experimental because some .task models can fail during prefill. Apple Intelligence or ExecuTorch is preferred on iOS.

Known Issues

• ExecuTorch is native-only and is not available on web.
• iOS ExecuTorch requires linking the ExecuTorch SwiftPM products in the app target, and that app target must support iOS 17 or newer.
• Apple Intelligence requires iOS 26.0 or later and a supported device.
• Android requires minSdkVersion 24 or higher.
• Model files are large, so production apps should usually download them after install.