tree-sitter-language-pack

Rust core library providing access to 305 tree-sitter parsers with on-demand download and caching support.

Installation

cargo add tree-sitter-language-pack

Quick Start

use tree_sitter_language_pack::{get_language, get_parser, available_languages};

// Initialize and download specific languages (optional)
tree_sitter_language_pack::init(&["python", "javascript", "rust"]).unwrap();

// Get a language (auto-downloads if needed)
let lang = get_language("python").unwrap();

// Get a pre-configured parser
let mut parser = get_parser("python").unwrap();
let tree = parser.parse("def hello(): pass", None).unwrap();
println!("{}", tree.root_node().to_sexp());

// List all available languages
for lang in available_languages() {
    println!("{}", lang);
}

// Process source code (auto-downloads language if needed)
let config = tree_sitter_language_pack::ProcessConfig::new("python").all();
let result = tree_sitter_language_pack::process("def hello(): pass", &config).unwrap();
println!("Functions: {}", result.structure.len());
println!("Imports: {}", result.imports.len());

// Pre-download languages for offline use
tree_sitter_language_pack::download(&["python", "javascript"]).unwrap();

// With chunking
let config = tree_sitter_language_pack::ProcessConfig::new("python").all().with_chunking(1000);
let result = tree_sitter_language_pack::process(source, &config).unwrap();
println!("Chunks: {}", result.chunks.len());

Features

Feature	Description
305+ Languages	Pre-compiled parsers for 305+ programming languages
On-Demand Downloads	Parsers are downloaded on-demand and cached locally for fast reuse
Selective Installation	Download only the languages you need; unused parsers never downloaded
Polyglot Bindings	Native bindings for Rust, Python, Node.js, Go, Java, Elixir, and C/C++
Automatic Caching	Downloaded parsers cached in platform-specific directories for offline use
CLI Tool	ts-pack download to pre-download parsers for offline/CI/Docker use

Downloading Parsers

Parsers are downloaded automatically on first use. For production, CI, or Docker, pre-download them:

// Download specific languages
tree_sitter_language_pack::download(&["python", "javascript", "rust"])?;

// Download all 305 languages
tree_sitter_language_pack::download_all()?;

Or use the CLI:

ts-pack download python javascript rust
ts-pack download --all
ts-pack download --groups web,systems

Static Compilation

To statically compile parsers into your binary (instead of downloading at runtime), set the TSLP_LANGUAGES environment variable at build time:

TSLP_LANGUAGES=python,rust,javascript cargo build

API Reference

Language Discovery

• available_languages() -- list all supported language names
• has_language(name) -- check if a language is available
• language_count() -- total number of supported languages

Language Detection

• detect_language(path) -- detect language from file path
• detect_language_from_extension(ext) -- detect language from a bare file extension
• detect_language_from_path(path) -- detect language from a file path
• detect_language_from_content(content) -- detect language from shebang line

Parsing

• get_parser(name) -- create a tree-sitter parser configured for a supported language

Download API

• init(languages) -- pre-download specific languages for offline use
• download(languages) -- download parsers on demand

Intelligence

• process(source, config) -- extract structured analysis (functions, classes, imports, comments, chunks) from source code

Syntax Highlighting Queries

• get_highlights_query(language) -- get bundled highlights.scm query for a language
• get_injections_query(language) -- get bundled injections.scm query
• get_locals_query(language) -- get bundled locals.scm query

For full documentation, see kreuzberg.dev.

License

MIT -- see LICENSE for details.

---

Part of tree-sitter-language-pack -- A comprehensive collection of tree-sitter language parsers with polyglot bindings.