Features

October 27, 2025 · View on GitHub

demo

podcast AI Podcast：https://podcast-997.pages.dev/ :)

# need GOOGLE_API_KEY in environment variables
# default use language English

# websit
python -m demo.content_parser_tts instruct-content-tts \
    "https://en.wikipedia.org/wiki/Large_language_model"

python -m demo.content_parser_tts instruct-content-tts \
    --role-tts-voices zh-CN-YunjianNeural \
    --role-tts-voices zh-CN-XiaoxiaoNeural \
    --language zh \
    "https://en.wikipedia.org/wiki/Large_language_model"

# pdf
# https://web.stanford.edu/~jurafsky/slp3/ed3bookaug20_2024.pdf 600 page is ok~ :)
python -m demo.content_parser_tts instruct-content-tts \
    "/Users/wuyong/Desktop/Speech and Language Processing.pdf"

python -m demo.content_parser_tts instruct-content-tts \
    --role-tts-voices zh-CN-YunjianNeural \
    --role-tts-voices zh-CN-XiaoxiaoNeural \
    --language zh \
    "/Users/wuyong/Desktop/Speech and Language Processing.pdf"

cmd chat bots:
- local-terminal-chat(be/fe)
- remote-queue-chat(be/fe)
- grpc-terminal-chat(be/fe)
- grpc-speaker
- http fastapi_daily_bot_serve (with chat bots pipeline)
- bots with config see notebooks:
  - Run chat bots with colab notebook 🏃
support transport connector:
- pipe(UNIX socket)
- TCP/IP socket
- gRPC
- queue
  - redis
  - zmq
  - rocketmq
chat bot processors:
- aggreators(llm use, assistant message),
- ai_frameworks
  - langchain: RAG
  - llamaindex: RAG
  - autoagen: multi Agents
- realtime voice inference(RTVI),
- transport:
  - WebRTC:
    - daily: audio, video(image)
    - livekit: audio, video(image)
    - agora: audio, video(image)
    - small_webrtc: audio, video(image)
  - WebSocket:
    - Websocket server
    - fastapi Websocket server
- ai processor: llm, tts, asr etc..
  - llm_processor:
    - openai(use openai sdk)
    - google gemini(use google-generativeai sdk)
    - litellm(use openai input/output format proxy sdk)
core module:
- local llm:
  - llama-cpp (support text,vision with function-call model)
    - llm_llamacpp_generator
  - fastdeploy:
    - llm_fastdeploy_vision_ernie4v
    - llm_fastdeploy_generator
  - tensorrt_llm:
    - llm_trtllm_generator
    - llm_trtllm_runner_generator
  - sglang:
    - llm_sglang_generator
  - vllm:
    - llm_vllm_generator
    - llm_vllm_vision_skyworkr1v
    - llm_vllm_deepseek_ocr, llm_office_vllm_deepseek_ocr
  - transformers(manual, pipeline) (support text; vision,vision+image; speech,voice; vision+voice)
    - llm_transformers_manual_vision_llama
    - llm_transformers_manual_vision_molmo
    - llm_transformers_manual_vision_qwen
    - llm_transformers_manual_vision_deepseek
    - llm_transformers_manual_vision_janus_flow
    - llm_transformers_manual_vision_janus
    - llm_transformers_manual_vision_smolvlm
    - llm_transformers_manual_vision_gemma
    - llm_transformers_manual_vision_fastvlm
    - llm_transformers_manual_vision_kimi
    - llm_transformers_manual_vision_mimo
    - llm_transformers_manual_vision_keye
    - llm_transformers_manual_vision_glm4v
    - llm_transformers_manual_vision_skyworkr1v
    - llm_transformers_manual_vision_deepseek_ocr
    - llm_transformers_manual_image_janus_flow
    - llm_transformers_manual_vision_janus
    - llm_transformers_manual_image_janus
    - llm_transformers_manual_speech_llasa
    - llm_transformers_manual_speech_step
    - llm_transformers_manual_voice_glm
    - llm_transformers_manual_vision_voice_minicpmo, llm_transformers_manual_voice_minicpmo,llm_transformers_manual_audio_minicpmo,llm_transformers_manual_text_speech_minicpmo,llm_transformers_manual_instruct_speech_minicpmo,llm_transformers_manual_vision_minicpmo
    - llm_transformers_manual_qwen2_5omni, llm_transformers_manual_qwen2_5omni_audio_asr,llm_transformers_manual_qwen2_5omni_vision,llm_transformers_manual_qwen2_5omni_speech,llm_transformers_manual_qwen2_5omni_vision_voice,llm_transformers_manual_qwen2_5omni_text_voice,llm_transformers_manual_qwen2_5omni_audio_voice
    - llm_transformers_manual_kimi_voice,llm_transformers_manual_kimi_audio_asr,llm_transformers_manual_kimi_text_voice
    - llm_transformers_manual_vita_text llm_transformers_manual_vita_audio_asr llm_transformers_manual_vita_tts llm_transformers_manual_vita_text_voice llm_transformers_manual_vita_voice
    - llm_transformers_manual_phi4_vision_speech,llm_transformers_manual_phi4_audio_asr,llm_transformers_manual_phi4_audio_translation,llm_transformers_manual_phi4_vision,llm_transformers_manual_phi4_audio_chat
    - llm_transformers_manual_vision_speech_gemma3n,llm_transformers_manual_vision_gemma3n,llm_transformers_manual_gemma3n_audio_asr,llm_transformers_manual_gemma3n_audio_translation
    - llm_transformers_manual_voice_step2, llm_vllm_client_step_audio2, llm_vllm_client_step_audio2_mock
    - llm_transformers_manual_qwen3omni, llm_transformers_manual_qwen3omni_vision_voice
- remote api llm: personal-ai(like openai api, other ai provider)
AI modules:
- functions:
  - search: search,search1,serper
  - weather: openweathermap
- punctuation:
  - punc_ct_tranformerm, punc_ct_tranformer_offline, punc_ct_tranformer_onnx, punc_ct_tranformer_onnx_offline
- speech:
  - asr:
    - whisper_asr, whisper_timestamped_asr, whisper_faster_asr, whisper_transformers_asr, whisper_mlx_asr
    - whisper_groq_asr
    - sense_voice_asr
    - minicpmo_asr (whisper)
    - qwen2_5omni_asr (whisper)
    - kimi_asr (whisper)
    - vita_asr (sensevoice-small)
    - phi4_asr (conformer)
    - gemma3n_asr (matformer)
  - asr_live:
    - asr_streaming_sensevoice
  - speech enhancement:
    - enhancer_ans_rnnoise
    - enhancer_ans_dfsmn
  - audio_stream: daily_room_audio_stream(in/out), pyaudio_stream(in/out)
  - detector: porcupine_wakeword,pyannote_vad,webrtc_vad,silero_vad,webrtc_silero_vad,fsmn_vad
  - player: stream_player
  - recorder: rms_recorder, wakeword_rms_recorder, vad_recorder, wakeword_vad_recorder
  - tts:
    - tts_edge
    - tts_g
    - tts_coqui
    - tts_chat
    - tts_cosy_voice,tts_cosy_voice2
    - tts_f5
    - tts_openvoicev2
    - tts_kokoro,tts_onnx_kokoro
    - tts_fishspeech
    - tts_llasa
    - tts_minicpmo
    - tts_zonos
    - tts_step
    - tts_spark
    - tts_orpheus
    - tts_mega3
    - tts_vita
  - vad_analyzer:
    - daily_webrtc_vad_analyzer
    - silero_vad_analyzer
    - ten_vad_analyzer
  - turn_analyzer
    - v2_smart_turn_analyzer
- vision
  - OCR(Optical Character Recognition):
    - PaddleOCR
    - GOT(the General OCR Theory)
  - Detector:
    - YOLO (You Only Look Once)
    - RT-DETR v2 (RealTime End-to-End Object Detection with Transformers)
gen modules config(*.yaml, local/test/prod) from env with file: .env u also use HfArgumentParser this module's args to local cmd parse args
deploy to cloud ☁️ serverless:
- vercel (frontend ui pages)
- Cloudflare(frontend ui pages), personal ai workers
- fastapi-daily-chat-bot on cerebrium (provider aws)
- fastapi-daily-chat-bot on leptonai
- fastapi-daily-chat-bot on modal
- aws lambda + api Gateway
- docker -> k8s/k3s
- etc...