README.md

September 20, 2026 · View on GitHub

Fast, cheap, structured decisions baked into your Rails app's control flow.

Install · Decisions · Guardrails · Router · Testing

class SupportTriage < Jevalyn::Decision
  question :department, type: :choice,
    instructions: "Which team should handle this?",
    criteria: {
      billing:   "Payments, invoicing, refunds",
      technical: "Bugs, outages, integrations",
      sales:     "Pricing, upgrades, new accounts"
    }

  confidence_threshold 0.75
end

result = SupportTriage.evaluate(ticket.body)

result.department   # => :technical
result.certain?     # => true

Jevalyn is the decision layer for your Rails app, built on Jev. That is a routing decision made by a model, inside a Rails request, in about as long as a database query. Not a prompt, not a parsed JSON blob, not a retry loop around something that might return prose this time.

What this is, and what it is not

Jevalyn wraps TypeSafe's Jev, a System One model. You give it a state and a set of typed questions; it gives back typed, calibrated answers. There are exactly three kinds of question:

Type	Ask it	Get back
`noul`	a yes/no question	the probability the answer is yes, `0.0`–`1.0`
`choice`	pick one of up to 255 options you define	the winner, the full distribution, a confidence
`score`	rate against 2–10 ordered levels you define	a weighted score, the distribution, a confidence

Jev does not generate text. No summaries, no drafts, no open-ended reasoning, no tool calls. If you need prose, you need an LLM, and Jevalyn will happily route to one (see Router) — but it will not pretend to be one.

What it is good at is the decision around the work: which queue does this belong in, is this safe to auto-approve, how severe is this, does this comment need a human, which of these 200 documents actually answers the question. Those are cheap, fast, and type-safe here, and expensive, slow, and stringly-typed anywhere else.

You can build anything on top of it — trade screening, recipe filtering, content moderation. "Use Jevalyn for recipes" means classifying and scoring recipes, not writing them.

Install

# Gemfile
gem "jevalyn"

$ bundle install
$ bin/rails g jevalyn:install

That writes config/initializers/jevalyn.rb and creates app/decisions/. Then put your key in the environment:

$ export TYPESAFE_API_KEY=ts_...
$ bin/rails jevalyn:ping

Get a key at typesafe.ai. Jev bills on input tokens only — output tokens are free — at roughly $0.042 per million input tokens as of jev-1.13. A support ticket costs a fraction of a cent to triage. Check the models page for current pricing and rate limits; they are still moving.

Decisions

A Jevalyn::Decision is a named set of questions your app asks about a piece of state.

$ bin/rails g jevalyn:decision SupportTriage urgent:noul department:choice severity:score

class SupportTriage < Jevalyn::Decision
  question :urgent, type: :noul,
    instructions: "Does this convey urgency?",
    criteria: { true: "Explicitly time-sensitive", false: "No urgency expressed" }

  question :department, type: :choice,
    instructions: "Which team should handle this?",
    criteria: {
      billing:   "Payments, invoicing, refunds",
      technical: "Bugs, outages, integrations",
      sales:     "Pricing, upgrades, new accounts"
    }

  question :severity, type: :score,
    instructions: "How severe is this issue?",
    criteria: ["trivial", "minor", "major", "critical"]

  confidence_threshold 0.75
end

Every question is validated when the class body runs, so a rubric with eleven score levels or a :choice with no criteria fails on boot — not as a 422 on a Friday afternoon.

All three questions go out in one request. Jev reads the state once and evaluates every question against it in parallel, which is both cheaper and faster than asking three times. Add speculative questions freely; they cost a few tokens each.

Reading the result

result = SupportTriage.evaluate(ticket.body)

result.urgent               # => 0.92        the raw probability
result.urgent?              # => true        at the default 0.5 cutoff
result.urgent?(0.95)        # => false       your cutoff, your call

result.department           # => :technical
result.department_confidence     # => 0.82
result.department_probabilities  # => { "billing" => 0.08, "technical" => 0.85, ... }
result.department_answer.runner_up  # => :billing

result.severity             # => 2.4         weighted, lands between levels
result.severity_label       # => "major"     nearest level
result.severity_level       # => 2

result.certain?             # => true        every answer cleared 0.75
result.uncertain_questions  # => []          or the names that did not
result.model                # => "jev-1.13.0"
result.input_tokens         # => 312
result.values               # => { urgent: 0.92, department: :technical, severity: 2.4 }

Two things here differ from what you might expect, and both come straight from the API:

A score is a Float, not a label. 2.4 means past major and heading for critical. Use severity_label when you want the nearest level's name, and severity itself when you want to do arithmetic.
A noul has no confidence. Jev returns confidence on Choice and Score answers only — for a noul, the probability is the answer, and 0.5 is the model telling you it does not know. Jevalyn fills the gap with certainty, derived from how far the value sits from a coin flip, so certain? works uniformly across all three types. That number is Jevalyn's arithmetic, not TypeSafe's; confidence stays nil so you always know which is which.

Confidence

Confidence is the second axis. The answer tells you what; confidence tells you whether to act.

result = RefundDecision.evaluate(request)

if result.uncertain?(0.9)
  HumanReview.enqueue(request)     # the model said "I'm not sure"
elsif result.approve?
  Refund.issue(request)
end

A threshold is not one number for your whole app. Misrouting a support ticket is recoverable; auto-approving a refund is not. Set confidence_threshold per decision, and override at the call site when the stakes change:

SupportTriage.evaluate(body, confidence_threshold: 0.95)

State

Anything JSON-shaped works: a String, a Hash, an Array, an ActiveRecord model.

SupportTriage.evaluate(ticket.body)                        # a string
SupportTriage.evaluate({ subject: s, body: b, plan: "pro" }) # a hash
SupportTriage.evaluate(conversation.messages)              # an array
SupportTriage.evaluate(ticket)                             # an AR model

For a model, define #jevalyn_state and send only what the decision needs. The default is as_json, which ships every column — wasteful in tokens and careless with data that did not need to leave the building.

class Ticket < ApplicationRecord
  def jevalyn_state
    { subject:, body:, plan: account.plan, previous_tickets: account.tickets.count }
  end
end

Jev's budget is 64k tokens per request, and 32k for the state plus the longest single question. SupportTriage.estimated_tokens(state) gives a rough count before you send.

Checking cost before you ship

SupportTriage.payload_for(ticket.body)   # exactly what would be sent, unsent
SupportTriage.estimated_tokens(ticket.body)

Guardrails

A Guardrail is a Decision narrowed to one job: should this be allowed through?

$ bin/rails g jevalyn:guardrail ToolCall

class ToolCallGuardrail < Jevalyn::Guardrail
  question :safe_to_execute, type: :noul,
    instructions: "Is this tool call safe to run without human review?",
    criteria: {
      true:  "Read-only, scoped to the current user's own data",
      false: "Writes, deletes, spends money, or touches another account"
    }

  allow_above 0.95
  on_error :deny
end

ToolCallGuardrail.check(tool_call).allow?

One noul question, and allow_above is the probability it must reach. The default is 0.5 — a coin flip — which is almost certainly not what you want in front of anything destructive.

If the API call itself fails, a guardrail denies rather than failing open:

result = ToolCallGuardrail.check(payload)

result.deny?      # => true
result.failed?    # => true   — denied because TypeSafe was unreachable
result.error      # => #<Jevalyn::OverloadedError ...>

Set on_error :raise if you would rather handle the outage yourself.

Router

A dispatch table with a confidence floor underneath it. This is where Jev hands off to something slower when it is not sure.

router = Jevalyn::Router.new(SupportTriage, on: :department) do |r|
  r.route :billing,   to: BillingInbox
  r.route :technical, to: ->(ticket, result) { Oncall.page(ticket, result.severity) }
  r.route :sales,     to: SalesInbox

  r.uncertain_below 0.75, to: HumanQueue
end

router.call(ticket)

Handlers are anything responding to #call, or another Jevalyn::Decision. They get (state, result) if they take two arguments and (state) if they take one.

The router checks at build time that every option your :choice can return has a route — so adding a fourth department and forgetting to route it is a boot error, not a production exception on an unusual ticket.

Without a decision it is a plain dispatch table:

router = Jevalyn::Router.new do |r|
  r.route :lookup, to: OrderLookup
  r.route :reason, to: llm_client
end

router.dispatch(:lookup, state: order)

Keep it thin. If a route needs branching logic, that logic belongs in the handler.

Testing

TypeSafe has no sandbox key, so mock_mode is entirely local: with it on, the client never opens a connection.

# spec/spec_helper.rb
require "jevalyn/testing/rspec"

That turns mock mode on for the suite, resets stubs between examples, and adds helpers:

it "routes a payment failure to the technical team" do
  stub_jevalyn(SupportTriage, urgent: true, department: :technical, severity: "major")

  expect(TriageJob.perform_now(ticket).queue).to eq("technical")
  expect(SupportTriage).to have_been_evaluated.once
end

Stubbed values are written the way you would assert on them — true, :technical, "major" — and expanded into a response the real API could have returned, probability distribution and all. An unstubbed evaluation raises, so a new question added to a decision surfaces in the suite rather than silently answering nil.

stub_jevalyn(SupportTriage, urgent: 0.61, ...)                 # exact probability
stub_jevalyn(SupportTriage, confidence: 0.4, ...)              # exercise the uncertain path
stub_jevalyn(SupportTriage) { |state| { urgent: state.include?("!") } }   # per-state
forbid_jevalyn(SupportTriage)                                  # assert it is never called

Minitest works the same way via require "jevalyn/testing/minitest".

Cassettes

For the handful of tests that should run against real answers, record once and replay:

jevalyn_cassette("spec/cassettes/triage.json") do
  result = SupportTriage.evaluate(File.read("spec/fixtures/payout_failure.txt"))
  expect(result.department).to eq(:technical)
end

The first run with a real key records; every run after replays. Requests are keyed by a digest of the exact body sent, so changing a rubric misses the cassette rather than replaying a stale answer against a question you no longer ask. Commit the file.

Testing against the live model

Stubs prove your code is right. They cannot prove your rubric is right — and a rubric is the part that drifts, both when you reword it and when jev-latest moves under you. Keep a handful of real inputs whose answer you are sure of, tag them :jevalyn_live, and run them deliberately:

it "recognises a payout outage as technical", :jevalyn_live do
  expect(SupportTriage.evaluate(payout_outage_ticket).department).to eq(:technical)
end

Background evaluation

Jev answers in well under a second, so the default advice is to call evaluate inline and keep the decision in your control flow. That is the whole point of a System One model — it is fast enough to be part of the request.

Use the queue when the decision is genuinely not on the critical path: backfills, batch scoring, or anywhere a third-party outage must not take a request down with it.

SupportTriage.evaluate_later(ticket, on: TicketRouter)
# => TicketRouter.call(result, ticket)

The handler is named rather than passed as a block, because a block cannot be serialised onto a queue.

Configuration

Jevalyn.configure do |c|
  c.api_key = ENV["TYPESAFE_API_KEY"]
  c.default_model = "jev-latest"
  c.timeout = 10
  c.open_timeout = 5
  c.max_retries = 2
  c.mock_mode = Rails.env.test?
end

jev-latest is an alias, and an alias moves when TypeSafe ships a release. Every Result reports the versioned model that actually answered (result.model), so log it. Once you have tuned thresholds against a version, pin it:

class RefundDecision < Jevalyn::Decision
  model "jev-1.13.0"
end

Errors

Error	When
`Jevalyn::ConfigurationError`	no API key, empty state, a malformed decision
`Jevalyn::InvalidQuestionError`	a rubric that cannot be sent — raised at class definition
`Jevalyn::AuthenticationError`	401
`Jevalyn::InvalidRequestError`	422, with the offending field in `#body`
`Jevalyn::RateLimitError`	429 — retried automatically
`Jevalyn::OverloadedError`	529 — retried automatically
`Jevalyn::TimeoutError`	the request did not complete
`Jevalyn::ConnectionError`	TypeSafe was unreachable

429, 529 and 5xx are retried with exponential backoff, honouring the API's retry-after header when it sends one. Everything else fails immediately, because it would fail identically on a retry.

Low confidence is not an error. It is the model doing its job. result.uncertain? is a branch in your code, not a rescue.

Generators

$ bin/rails g jevalyn:install
$ bin/rails g jevalyn:decision SupportTriage urgent:noul department:choice
$ bin/rails g jevalyn:guardrail ToolCall --allow-above 0.95

$ bin/rails jevalyn:ping         # check key, network, available models
$ bin/rails jevalyn:decisions    # list every Decision and Guardrail in the app