Jev Query Routing: Ask What It Means, Not Whether It's Right

I stopped asking the language model whether an answer was right. I asked it what the sentence meant.

That sounds like a smaller job. It turned out to be the useful one.

I came to this from the other direction. A couple of years ago, I led a search team at a large company. A few engineers and I spent a month or two building our own query-understanding model. We trained it to recognize whether someone was looking for help, checking inventory, or shopping.

We wrote thousands of hand-authored examples in JSONL, trained the model in Vertex AI, waited hours for each training run, and often waited until the next day to inspect the result and decide what to change. The work cost thousands of dollars before we had a model we could use. It was a serious investment in teaching software to classify what a person meant by a search.

About a year later, while building ChatSEO, I found myself trying to build a similar layer again. I wanted to understand a user's request, route it to the right API endpoint, decide what parameters to send, execute the call, then give the response and cached intent to a language model to analyze and explain.

Jev, TypeSafe's System One model, changed the shape of that design. It gives you this kind of classification out of the box, without building a labeled training set or waiting through training cycles, at a tiny fraction of the cost of the custom model work. Instead of asking a generative model to understand the request and plan every tool call, I could use Jev to resolve bounded routing choices. Then I could reserve the generative model for the part where it adds the most value: analyzing the API response and producing a useful answer.

More recently, I was building a system that answers questions from structured data. It writes a query, reads the rows, and turns them into a readable answer. Getting it to answer was the easy half. The hard half was knowing whether to trust what it said.

Those projects looked different—one routed a search, the other checked an answer—but they shared the same seam. People express intent in language; applications need explicit choices and values. Jev reads that meaning into typed judgments, then ordinary code takes over. The grader uses those judgments to decide what a sentence claims, while arithmetic decides whether its figures are true. The search router uses them to choose a function and its finite arguments, while application code calls the API. When prose needs interpretation or synthesis, a generative LLM can handle that separate job.

The first line of defense was deliberately boring. A grader pulls every figure out of an answer and checks it against the rows the system actually read: an exact value, a sum, a rate, a running total. If a claim can be reduced to arithmetic, no model gets a vote. That part works, and it hasn't changed.

The boundary: Jev can say what a sentence claims. Arithmetic decides whether its figure is true.

The trouble was the layer in front of the arithmetic. Before the grader could check a number, it had to know what the sentence around it meant. Was "about 100" a rounded measurement or a prediction? Was "raise the minimum to 40" a claim about the data, or an offer to change the query? If an answer used a nickname, which stored record did it mean?

The first version answered those questions with keyword lists. Every bug became another phrase added to a list. The lists got longer; the reading didn't get better.

A small model with a small job

Those are questions about language, so I handed them to Jev, TypeSafe's System One model, and gave it exactly one job. With a Choice, I give it some text and named options; it returns the selected option, a confidence, and probabilities across the options. With a Noul, I ask a binary question and get a probability for yes. It doesn't write the answer, it doesn't explain itself, and it never decides whether a number is right.

The model supplies an interpretation. Code supplies the verdict.

Take "about 100." The model classifies it as a rounded measurement rather than a forecast, and code checks the underlying value against the result rows. Two rules make that safe to lean on. First, the old keyword checks stay underneath as a fallback, so a missing or unsure reading degrades to the old behavior instead of turning into a guess. Second, wherever the model chooses, it chooses from candidates code built: the actual rows, the actual stored questions, or a fixed list of meanings. It can always say "none of these." It can't invent a row that isn't there.

That's the whole design. The model isn't an oracle; it's resolving an ambiguity code already bounded.

The same approach works on the question itself. A query can only be checked for the right time period, category, or minimum if someone has written down what the question implies, so the model reads the question too. One rule mattered more than any other here: if the model won't commit to a reading, that means "don't check," never "nothing was specified." If it can't tell whether the asker named their own minimum, the grader doesn't impose a default one. And an uncertain reading can raise a finding for a person to look at, but it can never fail a run on its own.

The first pass surfaced a handful of real findings. The most useful one wasn't a model problem at all: the grader had been misreading a perfectly good query, and that same misreading had been quietly failing correct answers.

Let Jev choose a function and fill its arguments

The same idea can power a search-API orchestration layer. The most useful TypeSafe pattern here is function calling: describe the operations your application already knows how to execute, then ask Jev to choose the right operation and fill its arguments from the values those functions accept.

Imagine a small commerce API with ordinary typed functions:

from typing import Literal

def search_help_articles(
    query: str = "",
    topic: Literal["account", "orders", "payments", "shipping"] | None = None,
) -> list[dict]: ...

def search_inventory(
    query: str = "",
    category: Literal["clothing", "equipment", "accessories"] | None = None,
    in_stock_only: bool = False,
) -> list[dict]: ...

def search_products(
    query: str = "",
    category: Literal["clothing", "equipment", "accessories"] | None = None,
    sort: Literal["relevance", "price_low_to_high", "price_high_to_low"] = "relevance",
) -> list[dict]: ...

def ask_for_clarification() -> None: ...

These functions—not Jev—own API execution. Their type signatures define the legal values. The route and each closed-set argument become typed questions: which function fits, which category, which topic, which sort order. Add plain-language descriptions to explain what each function and value means to the user. Keep open-ended values such as the search query as user text; don't pretend a finite Choice is a general-purpose text extractor.

Install the TypeSafe SDK and the cookbook's dispatcher helper (cooksafe), then give Dispatcher your functions, a TypeSafe client, and a small plain-language spec. The helper inspects the function signatures to find closed sets (Literal), sets of values, and boolean flags. The spec describes the functions and their arguments, including what the options mean in language users might use. Jev selects a function and fills its closed-set arguments in one call. Free text, numbers, and dates are not guessed; the function's default stands unless you add a separate extraction path.

For an API search, the spec describes each endpoint wrapper and each finite argument. Optional choices need a companion stated judgment so code can distinguish “the user asked for this filter” from “the user didn't mention it.” Boolean arguments are handled as flags. The dispatcher can ask branch-specific questions speculatively: Jev evaluates every question against the same request independently and in parallel, and code uses only the answers for the selected function.

For example, the search_products portion of a spec can look like this:

{
  "description": "Find products matching the user's request",
  "arguments": {
    "category": {
      "question": "Which category does the user mean?",
      "stated": "Does the user specify a product category?",
      "options": {
        "clothing": "Clothing and apparel",
        "equipment": "Sports or activity equipment",
        "accessories": "Accessories and add-ons"
      }
    },
    "sort": {
      "question": "How should the products be sorted?",
      "stated": "Does the user request a price order?",
      "options": {
        "relevance": "Most relevant first",
        "price_low_to_high": "Lowest price first",
        "price_high_to_low": "Highest price first"
      }
    }
  }
}

The function signature constrains what Jev can return; the spec teaches it how those values correspond to what the user said.

Now try: “Find in-stock equipment for wet trails.” The dispatcher can select search_inventory, choose equipment, and set in_stock_only to true. The application keeps “wet trails” as the original search text, adds it to the selected function's arguments, validates the request, and calls the API. The wrapper can translate the boolean into whatever filter the underlying API expects. If the results need explanation, a generative LLM can summarize those returned products. Jev never writes a query string, invents a price, or calls an endpoint itself.

The handler stays ordinary code. Here is the flow using the cookbook's Dispatcher shape:

from cooksafe import Dispatcher
from typesafe_sdk import TypeSafeClient

TOOLS = {
    "search_help_articles": search_help_articles,
    "search_inventory": search_inventory,
    "search_products": search_products,
    "ask_for_clarification": ask_for_clarification,
}

client = TypeSafeClient()
dispatcher = Dispatcher(SPEC, TOOLS, client)

call = dispatcher(user_query)
if (
    call.name == "ask_for_clarification"
    or call.confidence < ROUTE_CONFIDENCE_FLOOR
):
    return ask_user_to_clarify()

# The dispatcher fills the closed-set arguments. Keep the original free-text query
# in application code and pass it to the selected search function yourself.
args = {
    name: argument.value
    for name, argument in call.arguments.items()
    if not argument.omitted
}
args["query"] = user_query
result = TOOLS[call.name](**args)
return summarize_if_needed(user_query, result)

Here, call.name is the selected function, call.arguments contains the selected closed-set arguments, and call.confidence is the weakest confidence among the judgments in the call. The functions can be thin API adapters: their Literal parameters define the values Jev may select, and their query parameter receives the original user text from the application. If you prefer, a wrapper can bind that query before running the selected function.

The ROUTE_CONFIDENCE_FLOOR above is application policy, not a universal default. A Choice's confidence describes how concentrated its probability distribution is; it is not a guarantee that the function call is correct. Calibrate it on representative requests, including ambiguous queries and requests that should select none. Keep the API key on the server, and execute only a registered function with validated arguments.

This is not a reason to replace every trained model or every LLM call. Jev is a strong fit when you can describe the decision as a small set of meaningful options or typed arguments. For a refined search phrase, code can find candidate spans and use a Jev Choice to select the one that matches the user's intent; numbers and dates still need extraction and validation or a clarification. Jev also doesn't write the nuanced explanation that may be needed after an API call. Use it where its shape fits, batch independent judgments together, and make a second request only when the next judgment depends on evidence returned by the API. Measure the whole workflow, including uncertainty, latency, and the cost of a wrong route.

This is where Jev changes the architecture I had been trying to build with ChatSEO. The orchestration layer can be code plus typed judgments: select the function, fill its closed-set arguments, call the API. A generative LLM can then focus on analyzing the response and explaining it when the result actually needs synthesis.

For the full pattern—including deriving questions from typed function signatures, writing a plain-language spec, and dispatching the result—see TypeSafe's function-calling cookbook. The key design rule remains the same: define the actions and legal choices in code; let Jev interpret the request; let code decide what to execute.

The cache that understood a rephrase

The clearest payoff came from caching.

Answers were cached by their exact wording, so two questions with the same intent paid for two full runs. Fuzzy text matching looked like the obvious fix, and on this data it was backwards. A genuine rephrase often scored as less similar than the same question about a different person or a different category. The wrong neighbors ranked above the right ones, so no similarity cutoff could separate them.

Now similarity only builds a shortlist, and the model picks one entry from it, or none. Choosing the gate on that pick took three tries, and the two failures are the most useful part:

1. Gating on the model's overall confidence rejected every true match. Confidence measures how concentrated the answer is, not whether it's right. A correct answer split 0.82 against 0.18 between two options scored only 0.65 on confidence.
2. Gating on the winner's probability rejected a correct answer split 0.51 / 0.29, with 0.19 on "none." The cache holds near-duplicates, so two equally good entries split the vote. Either one would have been fine, but a bar high enough to be safe on the winner turned both away.
3. Gating on one minus the probability of "none" asked the question that actually mattered: how sure is the model that some stored answer fits? That same split scores 1 − 0.19 = 0.81 and passes. And it refused the worst false match we'd seen: a total for one subset of records matched against a stored question about the overall total, which the model had picked at 0.71.

Same answers, different question. Only the last gate asks whether some stored answer fits.

Most rephrasings now replay, every near-miss we tried is refused, and a hit comes back in about half a second instead of nine seconds.

Prose that arithmetic can't reach

The system also writes short narrative summaries from the same data, and the brief forbids anything the rows can't show: quotes, causes, motives, events the data never recorded. Every number in a summary was already checked. Nothing else was, and that's exactly what a reader would notice, because a made-up quote reads as well as a real one.

So every sentence gets a reading against the rows the writer actually queried: grounded, contradicted, or a particular kind of unsupported detail. On a summary written by hand, with seven fabrications planted and the answers known in advance, it caught all seven, each as the right kind, and left all six true sentences alone.

It also caught an error arithmetic can't see: a real figure credited to the wrong person. The number passes every check because it really is in the rows. It's just on someone else's record.

The test data had its own trap. Older summaries had been re-checked against a newer copy of the data, so the first pass flagged "errors" that were really stale evidence. The reading only earned trust once we tested it on a summary whose answers were known before the check ran.

We also compared the meaning-readings against the keyword rules on the same 120 stored answers. The clean-answer count moved from 109 to 110—not a dramatic overall leap. But on the twelve sentences already known to fool the keyword rules, Jev read all twelve as intended. That is the result I care about: a small overall change, concentrated exactly where the old approach had been brittle. It is one evaluation on one corpus, not a guarantee for another application.

The gain is small across the corpus and complete on the sentences the old rules already missed.

What didn't survive being counted

Two more uses looked obviously good and fell apart once measured. Declining out-of-scope questions before any work ran would have caught almost nothing the system wasn't already declining on its own, and the next candidates were questions it had answered well. Choosing between ambiguous names automatically sounded helpful, but the few times ambiguity came up, asking "which one?" was the right answer.

The other lesson from measuring was humbling: several checks passed even with the bug put back in. The only proof a check works is reintroducing the defect and watching it go red.

The rule

A model should resolve an ambiguity that code prepared. It should never become the source of truth for a fact code can verify.