AITuberKit

All-in-One Toolkit for Building AI Characters

Notice: This project has adopted a custom license from version v2.0.0 onwards. If you are using it for commercial purposes, please check the Terms of Use section.

日本語｜ 简体中文｜ 繁體中文｜ 한국어｜ Polski

Overview

AITuberKit is an open-source toolkit that allows anyone to easily build a web application for chatting with AI characters. It supports a wide range of AI services, character models, and voice synthesis engines, with high customization options centered around dialogue and AITuber streaming functionality, along with various extension modes.

For detailed usage and configuration instructions, please visit the Documentation Site.

Main Features

1. Interaction with AI Characters

• Easy conversation with AI characters using API keys for various LLMs
• Multimodal support for recognizing camera footage and uploaded images to generate responses
• Retention of recent conversations as memory
• RAG-based long-term memory that utilizes past conversations as context

2. AITuber Streaming

• Retrieves YouTube stream comments for automatic responses from AI characters
• Choose between YouTube API / OneComme (WanKome) as comment source
• Conversation continuation mode allows spontaneous speech even without comments
• Customizable comment retrieval interval and user display name

3. Demo Terminal & Digital Signage

• Demo Terminal Mode: Fullscreen display for digital signage. Supports passcode authentication, NG word filter, and input length restrictions
• Presence Detection: Automatic visitor detection via camera face detection. Supports automatic greeting and farewell phrase playback
• Idle Mode: Characters speak automatically when conversation pauses. Supports three sources: fixed phrases, time-based greetings, and AI-generated content

4. Advanced Dialogue Modes

• Realtime API: Low-latency dialogue and function execution using OpenAI's Realtime API
• Audio Mode: Natural voice dialogue utilizing OpenAI's Audio API features
• Reasoning Mode: Display AI's thinking process and configure reasoning parameters
• Game Commentary Mode: Periodically analyzes screen captures and lets the AI character automatically generate and speak commentary matched to the game screen

5. Integration & Extension

• External Integration Mode: Connect with server applications via WebSocket for advanced functionality including text and image exchange
• Slide Mode: Mode where AI characters automatically present slides
• Message Reception Function: Accept text and images from external sources through a dedicated API to make AI characters speak

Supported Models & Services

Character Models

• 3D Models: VRM files (supports pose and gesture control via motion tags)
• 2D Models: Live2D files (Cubism 3 and later)
• Motion PNGTuber: Video-based character display (MotionPNGTuber)

Supported LLMs

• OpenAI
• Anthropic
• Google Gemini
• Azure OpenAI
• Groq
• Cohere
• Mistral AI
• Perplexity
• Fireworks
• LM Studio
• Ollama
• Dify
• xAI
• DeepSeek
• OpenRouter

Supported Voice Synthesis Engines

• VOICEVOX
• Koeiromap
• Google Text-to-Speech
• Style-Bert-VITS2
• AivisSpeech
• Aivis Cloud API
• Cartesia
• GSVI TTS
• ElevenLabs
• OpenAI
• Azure OpenAI

Quick Start

Development Environment

• Node.js: 24.x
• npm: ^11.6.2

Installation Steps

1. Clone the repository locally.

git clone https://github.com/tegnike/aituber-kit.git

2. Open the folder.

cd aituber-kit

3. Install packages.

npm install

4. Create a .env file as needed.

cp .env.example .env

5. Start the application in development mode.

npm run dev

6. Open the URL: http://localhost:3000

Quick Launch

After the initial setup is complete, you can launch by simply double-clicking the launch script.

• Windows: Double-click LAUNCH.bat
• macOS: Double-click LAUNCH.command (if you don't have execution permission, run chmod +x LAUNCH.command)

For detailed configuration and usage instructions, please visit the Documentation Site.

Running with Docker

1. Create a .env file.

cp .env.example .env

2. Start with Docker Compose.

docker compose up -d

3. Open the URL: http://localhost:3000

To stop:

docker compose down

Deployment

Vercel

1. Create a Vercel account and import your GitHub repository.

2. Set environment variables in the Vercel dashboard. Add the necessary API keys (e.g., OPENAI_API_KEY). Refer to .env.example for available environment variables.

3. Deployment is automatically executed when pushing to the Production Branch in your project settings (default is the main branch).

Cloudflare Workers

Deployment to Cloudflare Workers is supported. Use OpenNext to run the Next.js application on Cloudflare Workers.

1. Install the Wrangler CLI and log in to your Cloudflare account.

2. Change the project name in wrangler.jsonc as needed.

3. Set environment variables.

   • Frontend settings (NEXT_PUBLIC_*): Write them in the .env file. They are embedded in the client code at build time.
   • Server-side API keys (OPENAI_API_KEY, etc.): Set them as Wrangler secrets.

cp .env.example .env
# Edit .env to set NEXT_PUBLIC_* values

# Set server-side API keys as Wrangler secrets
npx wrangler secret put OPENAI_API_KEY

Repeat wrangler secret put for each required API key.

4. Verify operation with local preview.

npm run preview:cloudflare

5. Deploy to production.

npm run deploy:cloudflare

Notes:

• NEXT_PUBLIC_RESTRICTED_MODE=true is automatically set at build time, disabling file system APIs. Asset lists are provided from manifests generated at build time.
• Files exceeding 25MB or with non-ASCII filenames under public/ are automatically excluded from deployment.

⚠️ Important Security Notice

This repository is intended for personal use and development in local environments, as well as commercial use with appropriate security measures. However, please note the following when deploying to a web environment:

• API Key Handling: The system is designed to call AI services (OpenAI, Anthropic, etc.) and TTS services via a backend server, so proper management of API keys is necessary.
• Server-side secret protection: A security update requires AITUBERKIT_SERVER_SECRET_ACCESS_MODE when public API routes use server-side secrets such as OPENAI_API_KEY or CUSTOM_API_*, chat log saving, or server-side TTS URLs. Without this setting, some APIs return 403. See the v2.53.0 release notes for details and migration steps.

For Production Use

When using in a production environment, we recommend one of the following approaches:

1. Backend Server Implementation: Manage API keys on the server side to avoid direct API access from clients
2. Appropriate Explanation to Users: If users are using their own API keys, explain security considerations to them
3. Access Restriction Implementation: Implement appropriate authentication and authorization mechanisms as needed

Sponsorship

We are seeking sponsors to continue development.
Your support greatly contributes to the development and improvement of AITuberKit.

Contributors (in order of support)

Plus multiple private sponsors

Star History

Acknowledgments

This project was developed as a fork of ChatVRM published by pixiv Inc. We deeply appreciate pixiv Inc. for publishing such a wonderful open-source project.

Contributing

Thank you for your interest in contributing to the development of AITuberKit. We welcome contributions from the community.

Reporting Issues

If you find a bug or have an idea for a new feature, please let us know through the Issues page on GitHub.

When creating an issue, including the following information will help us respond smoothly:

• Detailed description of the problem or new feature
• Steps to reproduce (for bugs)
• Expected behavior vs. actual behavior
• Environment details (browser, OS, Node.js version, etc.)
• Screenshots or videos (if possible)

Pull Requests

If you want to improve code or add new features, please make changes in your forked repository and create a pull request.

• Focus on one feature or fix per pull request.
• Please describe the changes and the reason for them in the pull request description.
• Always set the target branch to develop.
• Don't worry about conflicts - the development team will handle them.

Terms of Use

License

This project has adopted a custom license from version v2.0.0 onwards.

• Free Use

  • Free for personal use, educational purposes, and non-profit purposes that are not for commercial purposes.

• Commercial License

  • A separate commercial license is required for commercial use.
  • For details, please check About the License.

Others

• Logo Usage Terms
• VRM and Live2D Model Usage Terms

Priority Implementation

This project accepts paid priority implementation of features.

• Features requested by companies or individuals can be implemented with priority.
• Implemented features will be published as part of this OSS project.
• Fees are individually quoted based on the complexity of the feature and the time required for implementation.
• This priority implementation is separate from the commercial license. If you want to use the implemented features for commercial purposes, you need to obtain a commercial license separately.

For details, please contact support@aituberkit.com.