README.md

May 20, 2026 · View on GitHub

codetracer-ruby-recorder

A recorder of Ruby programs that produces CodeTracer traces.

Warning

Currently it is in a very early phase: we're welcoming contribution and discussion!

Two recorders, by design

This repository ships two gems with different roles:

gems/codetracer-ruby-recorder/ — the production recorder. Rust native extension; emits CTFS v3 binary trace bundles (<prog>.ct). Use this one in real deployments.
gems/codetracer-pure-ruby-recorder/ — a pure-Ruby reference implementation that emits the legacy 3-file JSON shape (trace.json, trace_metadata.json, trace_paths.json). Kept JSON-only on purpose: the test suite runs every test program through both recorders, normalises the native recorder's CTFS output via ct print --json-events, and compares both against the same fixtures. The pure recorder is the independent oracle that keeps the native recorder honest. Do not migrate it to CTFS without coordinating with the test framework. See gems/codetracer-pure-ruby-recorder/README.md for the full rationale.

Installing as a gem

gem install codetracer-ruby-recorder

The command downloads a prebuilt native extension when available and falls back to building it from source. If this fails, install the pure Ruby version:

gem install codetracer-pure-ruby-recorder

After installing, load the tracer:

require 'codetracer_ruby_recorder' # native implementation
# require 'codetracer_pure_ruby_recorder' # pure Ruby implementation

recorder = RubyRecorder.new(Dir.pwd)
recorder.enable_tracing
# ... your code ...
recorder.flush_trace

The native recorder writes a single CTFS trace bundle (*.ct) per codetracer-specs/Recorder-CLI-Conventions.md §4 (CTFS-only). There is no --format flag: to inspect a recorded trace as JSON or human-readable text, run ct print (shipped with codetracer-trace-format-nim) on the produced *.ct file.

ruby gems/codetracer-ruby-recorder/bin/codetracer-ruby-recorder [--out-dir DIR] <path to ruby file> [-- <program args>]
# Writes a `*.ct` CTFS bundle into DIR (defaults to ./, or
# $CODETRACER_RUBY_RECORDER_OUT_DIR when set).
# Use `--` to pass arguments to the traced program.
# Pass --help to list all options.

The pure-Ruby fallback (no native extension) preserves the legacy 3-file JSON output shape and is intended for environments where the Rust native extension cannot be built:

ruby gems/codetracer-pure-ruby-recorder/bin/codetracer-pure-ruby-recorder [--out-dir DIR] <path to ruby file> [-- <program args>]
# Produces trace.json / trace_metadata.json / trace_paths.json in DIR.

Example recording a CTFS trace while passing arguments to the program:

ruby gems/codetracer-ruby-recorder/bin/codetracer-ruby-recorder --out-dir traces examples/runtime_code_execution.rb -- 2

# Inspect the resulting bundle:
ct-print --json traces/*.ct

however you probably want to use it in combination with CodeTracer, which would be released soon.

ENV variables

CODETRACER_RUBY_RECORDER_OUT_DIR — fallback for --out-dir. CLI flags always take precedence (convention §5).
CODETRACER_RUBY_RECORDER_DISABLED — set to 1 or true to skip recording entirely; the target script still runs (convention §5).
CODETRACER_RUBY_RECORDER_DEBUG=1 — enable additional debug-related logging.

There is no --format flag and no CODETRACER_FORMAT environment variable: the recorder is CTFS-only. Use ct print to convert the recorded *.ct trace to JSON or text for debugging or golden-snapshot fixtures.

Development Setup

This repository includes a comprehensive Ruby debugging setup for Visual Studio Code. See .vscode/README.md for detailed information about:

VS Code tasks for running and debugging Ruby code
Debug configurations for rdbg debugger
Ruby LSP integration for full language support
Interactive debugging with Pry

Quick Start for Contributors

Install debugging gems (if not using nix):
```
gem install debug pry
```
Use VS Code tasks: Open Command Palette (Cmd+Shift+P / Ctrl+Shift+P) and search for "Tasks: Run Task"
Debug with F5: Use the debug panel to set breakpoints and step through code
Run tests: Use the provided tasks or run manually:
```
ruby -I lib -I test test/test_tracer.rb
```

Future directions

The current Ruby support is a prototype. In the future, it may be expanded to function in a way similar to the more complete implementations, e.g. Noir.

Current approach: TracePoint API

Currently we're using the TracePoint API. This is very flexible and can function with probably multiple Ruby versions out of the box. However, this is limited:

it's not optimal
it can't track more detailed info/state, needed for some CodeTracer features(or for more optimal replays).

For other languages, we've used a more deeply integrated approach: patching the interpreter or VM itself (e.g. Noir).

Possible Alternative Approaches

Create a C extension for the VM, based on the `rb_add_event_hook2`

This would be a straight-forward port of the current code, but developed as a native extension (e.g. in C/C++ or Rust). The expected speedup will be significant.

Patching the VM

This approach may provide more depth: it can let us record more precisely calculated sub-expressions, assignments to record fields and other details required for the full CodeTracer experience.

The patching can be done either directly in the source code of the VM or through a binary instrumentation framework, such as Frida. The existing ruby-trace project provides an example for this.

Filtering

It would be useful to have a way to record only certain intervals within the program execution, or certain functions or modules: we plan on expanding the trace format and CodeTracer's support, so that this is possible. It would let one be able to record interesting parts of even long-running or more heavy programs.

Contributing

We'd be very happy if the community finds this useful, and if anyone wants to:

Use and test the Ruby support or CodeTracer.
Cooperate with us on supporting/advancing the Ruby support of CodeTracer.
Provide feedback and discuss alternative implementation ideas: in the issue tracker, or in our discord.
Provide sponsorship, so we can hire dedicated full-time maintainers for this project.

For maintainer instructions, see MAINTAINERS.md.

Development workflow

Use just for common tasks:

just build-extension  # compile native extension
just test             # run the test suite
just format           # apply automatic formatting for all languages
just lint             # run all linters

# Language-specific tasks
just format-rust      # Rust code
just format-ruby      # Ruby code (requires rubocop)
just format-nix       # Nix files
just lint-rust        # Rust formatting check
just lint-ruby        # Ruby syntax check
just lint-nix         # Nix formatting check

Direct-storage upload (Enterprise on-prem)

For Enterprise on-prem deployments, the recorder produces a materialized CTFS bundle on the local filesystem and a separate codetracer-managed-upload direct-materialized-finalize invocation PUTs it directly to the customer's storage server, then posts a metadata-only finalize to codetracer-ci. Trace bytes never traverse the codetracer-ci control plane.

After the recorder produces the materialized artifact set:

codetracer-managed-upload direct-materialized-finalize \
  --storage-config /etc/codetracer/trace-storage.json \
  --recording-id "${SESSION_ID}" \
  --object-key-prefix "traces/${TENANT_ID}/${SESSION_ID}/ruby-direct" \
  --idempotency-key "${SESSION_ID}-ruby" \
  --artifact-dir "${RECORDER_OUT_DIR}" \
  --language ruby

The full data-path overview, the static-config schema, and the Enterprise lease lifecycle are documented at codetracer-specs/Observability-Platform/docs/direct-storage-data-path.md. The HTTP endpoint reference is at codetracer-ci/rewrite-docs/04-apis-events/http-api.md section 4.10.

End-to-end coverage: StoragePolicyModelTests.e2e_ruby_recorder_materialized_direct_upload_with_static_config (M38 slice 5B) and the M39 NixOS test codetracer-ci-rewrite-multitenant-infra-materialized-recorders-incus (live recorder running inside an Incus container per tenant).

Legal info

LICENSE: MIT