🌈 caramba

A substrate for A.I. research.

Caramba is a comprehensive machine learning research stack built to guide you through the entire lifecycle of A.I. development. Whether you want to rapidly prototype a new concept or dive deep into low-level hardware optimization, Caramba provides a dedicated environment tailored to your exact workflow.

Operating on the core philosophy that a manifest is a model, Caramba allows you to define complex architectures entirely via YAML files instead of writing code. It acts as a complete laboratory, seamlessly taking you from your initial idea to a heavily optimized, fully profiled pipeline.

Core Capabilities:

• 📝 Fully Manifest-Driven: Declare your topology in simple YAML. Caramba goes far beyond standard layers and operations, allowing you to easily express sophisticated, non-standard mathematical primitives and advanced custom architectures.
• 🚀 Flexibility & Optimization: Dedicated to high performance, Caramba gives you the tools to choose your level of abstraction. Move fast to iterate on high-level ideas, or drop down for granular, low-level control over compute and memory.
• 🔬 Sophisticated Inspection: Deeply understand your network's behavior. Caramba is equipped with advanced inspection and profiling tooling, bringing clarity to every step of the end-to-end research process.
• 🤖 Integrated A.I. Collaboration: Supercharge your workflow with a built-in A.I. assistant and virtual research team. Caramba is built from the ground up to support both human team collaboration and agentic brainstorming.
• 🔐 Zero-Compromise Privacy: Working with proprietary or sensitive data? Caramba can operate entirely in an optional "local-only" mode, ensuring your research and intellectual property never leave your secure environment.

✨ Features

• Compute Primitives
  • Activation (ReLU, LeakyReLU, SELU, Sigmoid, Tanh, GeLU, Swish, SwiGLU)
  • Attention (SDPA, MQA, GQA, sliding window, softmax)
  • Convolution (Conv1D, Conv2D, Conv3D, ConvTranspose2D)
  • Embedding (token, RoPE, ALiBi, tied)
  • Math (matmul, add/mul/pow/atan2/mod, exp/log, rmsnorm, layernorm, groupnorm, softmax, logsumexp, dropout, sin, cos)
  • Pooling (avg, max, adaptive avg, adaptive max)
  • Projection (linear, fused QKV, tied embedding)
  • Shape (reshape, transpose, gather, scatter, where, masked fill, concat, split, view_as_heads, merge_heads, last_token, nearest upsample)
  • Masking (causal mask, apply mask)
  • Active Inference (free energy, expected free energy, belief update, precision weighting)
  • Energy-Based Model Blocks (Boltzmann distribution, EBM free energy, Langevin step, contrastive phase)
  • Causal Inference (do-calculus, backdoor, frontdoor, CATE, IV, counterfactual, DAG factorization)
  • Hawkes Process (intensity, kernel matrix, simulate, log-likelihood)
  • Markov Blanket (partition, mutual information, internal/active flow)
  • Predictive Coding (prediction, prediction error, representation/weight updates)
  • VSA (bind, bundle, permute, inverse permute, similarity)
• Multiple Compute Backends
  • CPU (Go native; amd64 and arm64 only — 32-bit GOARCH=386 is not supported)
  • SIMD/Assembly
    • AVX-512 (amd64)
    • AVX2 (amd64)
    • SSE2 (amd64)
    • NEON (arm64)
  • CUDA
  • METAL
  • XLA
• Optimizers (SGD, Adam, AdamW, AdaMax, AdaGrad, AdaDelta, RMSProp, Lion, LARS, LAMB, L-BFGS, Hebbian)
• Training Models
• Fine-tuning Models
• Manifest Compiler (verify, canonicalize, CSE, algebraic simplify, fusion, DCE, memory planning, cost scheduling)
• SafeTensors architecture manifests (from_safetensors, config-driven registry lookup, direct tensor binding)
• Hugging Face Hub Asset Resolver (revision-pinned, content-addressed, Xet CAS)
• Provenance Ledger (signed by pkg/notary)
• Streaming Chat Runtime (KV cache, sampling, qpool startup events)
• Diffusion Pipeline (FlowMatch Euler, prompt encoder + denoiser + VAE decoder)
• Supported Pre-trained Models
  • openai-community/gpt2
  • meta-llama/Llama-3.2-1B-Instruct (Gated model, requires request for access)
  • meta-llama/Llama-4-Scout-17B-16E
  • black-forest-labs/FLUX.2-klein-4B
  • google/gemma-4-31B-it
  • ibm-granite/granite-4.1-8b
  • Qwen/Qwen3-Coder-Next
  • stabilityai/stable-diffusion-3-medium
  • meituan-longcat/LongCat-AudioDiT-3.5B
  • facebook/ijepa_vith16_1k
  • facebook/vjepa2-vitg-fpc64-256
• Visual Node-Graph Architecture Builder
• ModelScope deep inspection tools
• Layer Surgery tools
• Hyperparameter Tuner
• Distributed Training
• Integrated Benchmarking Suite
• Deeply Integrated A.I. Assistant and Research Team
• Ergonomic WYSIWYG LaTeX Paper Editor
• Multi-User/Team Collaboration

🚀 Quick start

go install github.com/theapemachine/caramba@latest

caramba serve

The above command brings up just the HTTP API, which means you still have to bring up the data stores yourself.

Alternatively you could grab the docker-compose.yml from this repository to make this process much easier.

caramba research <name> lays out a project as a directory under version control:

research/project/my-ablation-study/
├── manifest/
│   ├── architecture/   # the architectures under comparison
│   └── operation/      # custom operations specific to this study
└── paper/              # write-up

Configuration lives in cmd/asset/config.yml and is loaded through pkg/config. The resolver tries --config, then ./cmd/asset/config.yml, ./config.yml, $HOME/.caramba/config.yml, and finally the binary's embedded default. Data-store clients (pkg/store/*) read store.qdrant, store.neo4j, store.elasticsearch, and store.deeplake from that file (secrets may use ${...} expansion); they do not call os.Getenv directly.

→ Getting Started

Building for a specific backend

# CPU — Go + AVX2/SSE2/NEON. Always available.
go build ./pkg/backend/compute/cpu/...

# CUDA — Linux, NVIDIA CUDA toolkit
CGO_ENABLED=1 go build -tags "cgo cuda" ./pkg/backend/device/cuda/...

# Metal — macOS, Xcode command-line tools (darwin + cgo selects Metal automatically)
go generate ./pkg/backend/device/metal
CGO_ENABLED=1 go build -tags cgo ./pkg/backend/device/metal/...

# XLA via PJRT — configure compute.xla in cmd/asset/config.yml first
go build -tags "cgo xla" ./pkg/backend/device/xla/...

type Runner interface {
    Execute(
        ctx context.Context,
        graph *ir.Graph,
        targets []*ir.Node,
    ) (map[string]tensor.Tensor, error)

    Location() tensor.Location
    Close() error
}

Backend kernels upload values once into a resident tensor store and only download at real boundaries. The executor releases owned dependencies after their last graph consumer; the host arena reuses released spans. Host-staged dispatch is restricted to the host backend, so Metal, CUDA, and XLA paths cannot silently route through CPU slices.

→ Compute Backends

💾 Repository layout

cmd/                Cobra CLI: serve, chat, image, research
  asset/config.yml  The single config source
pkg/
  manifest/         YAML → IR compiler, registry, lowering
  runtime/          Manifest runtime programs, state, ops, schedulers, graph bridge
  backend/
    compute/        Runner interface + cpu/, cuda/, metal/, xla/
    api/            HTTP server
  hub/              Hugging Face cache, Xet CAS
  tokenizer/        ByteLevel BPE
  model/            Weight binding, SafeTensors loader
  notary/           Identity + provenance ledger
  store/            S3, Elasticsearch, Neo4j, Qdrant, DeepLake
  config/           Single config gateway
frontend/           Vite + React + Flume node editor
docs/               Long-form documentation
AGENTS.md           Backend implementation contract — required reading for kernel work

---

🔬 Testing

Every code file has a _test.go mirror. Tests are GoConvey-style ("Given X, it should Y", nested). Backend kernels run parity tests against the scalar reference at N ∈ {1, 7, 64, 1024, 8192} with tight ULP bounds — the tolerance is a contract, not a knob.

go test ./...
CGO_ENABLED=1 go test -tags cgo            ./pkg/backend/device/metal/...
CGO_ENABLED=1 go test -tags "cgo cuda"     ./pkg/backend/device/cuda/...
              go test -tags "cgo xla"      ./pkg/backend/device/xla/...

---

📓 Documentation

Document	What's inside
Getting Started	Install, first chat, first study
Architecture	System design, IR, executor
Manifest & Governance	Manifest grammar, compiler pipeline, examples
Compute Backends	CPU/SIMD, CUDA, Metal, XLA in depth
Backend inventory	device.Backend methods ↔ ir.RequiredOperationIDs()
CPU dispatch matrix	Per-domain scalar / AVX-512 / AVX2 / SSE2 / NEON registration
Device backend matrix	Metal / CUDA / XLA kernel registrations, dtypes, required-op coverage
Backend coverage matrix	Combined T1.2–T1.4 registration snapshot and R1 execution-target summary
Backend compliance audit	T1.6 machine checks: forbidden phrasing, cross-ISA calls, amd64 scalar tails, loose test epsilons
Operations	Operation library, SIMD kernels, custom ops
Frontend & Visualization	Node editor, microscope tooling
The Notary	Identity, ledger, custody model
Agents	Conversational ingress, LLM providers
AGENTS.md	Backend implementation contract for contributors

---

License

MIT