Take the last piece of operational automation you shipped with an LLM inside it and read the prompts with one question in mind. How many of them ask the model to choose? The triage bot that reads an alert and returns {"team": "database"}. The workflow that looks at a failed job and answers "flaky or real". The Terraform reviewer that tags a plan as "matches the PR" or "does not". The agent wrapper that asks a second model whether the kubectl command the first one proposed is safe to run. In the pipelines I review, that shape covers most of the calls. The model is asked to pick from a list the team already wrote. The paragraph it writes around the pick is thrown away by the next line of code.

What that costs is easy to list and hard to notice, because each item is small. A frontier model takes seconds to answer, so a bot that chains three of those is slower than the human it replaced. Each call is billed at frontier prices for one word of information. The JSON comes back wrapped in prose often enough that every team ends up with a parser that strips fences and silently defaults when it fails. And the confidence is text. "I am fairly confident this is a database issue" cannot be thresholded, compared across weeks or plotted against what the on-call engineer actually did.

My position is that those calls belong to a different class of model. Jev, from TypeSafe AI, is the first product I have seen built for that slot. It takes a state and a set of typed questions and returns a choice with a probability distribution and a confidence derived from it, in what the vendor says is about a hundred milliseconds, without generating a token of prose. The architecture that follows from it is a division of labor. Jev selects and classifies. The LLM investigates and produces. Code applies rules, validates and executes. And the only responsible way to put it into an operations pipeline is in observation mode, logging what the model would have decided next to what the team did.

One disclosure before the details. TypeSafe opened early access on September 15, 2026, went public on September 20 with five dollars of credit per account and paused signups two days later under what it called an "immense swell of demand". I was on the wrong side of that pause, so I have not run Jev against production traffic. Everything below is checked against the current API reference and the model's published failure modes. The seven cases are architecture designs to test. None of them is a production report. The companion repository runs them against an offline stub, with a --live flag for anyone who holds a key. The stub's numbers are not Jev's.

This article explains what Jev is and what it gives up, puts the map of cloud and DevOps uses on one sheet and says which of them the companion repository simulates, lays out the division of labor as a pattern and shows where each layer fails, walks through seven places in a cloud and SRE pipeline where a decision model fits, and closes with how to start in shadow mode plus a proof of concept you can run this week.

What Jev Is, and What It Gives Up

TypeSafe calls Jev a System One model, after the fast, intuitive mode of thinking Kahneman popularized. You send POST https://api.typesafe.ai/v1/systemone with a bearer token and a body holding three fields. state is the material to judge, a string, object or array of text. model is jev-latest, which today resolves to jev-1.13.0. questions is a map of named, typed questions, all evaluated against the same state in parallel, so adding one usually adds little latency. Three question types exist. noul is a yes/no question and returns noul, a number from 0 to 1, with no confidence field. choice takes a criteria map of up to 255 options, each with an optional description, and returns choice, probabilities across every option and a confidence. score takes an ordered criteria array of two to ten levels and returns a probability-weighted score, a legend, probabilities per level and confidence. The response also carries the versioned model that answered and token usage.

Some numbers, all from the vendor. Input costs 0.042 dollars per million tokens and output is free. Context is 64k tokens per request, 32k for the state plus the longest question. The documentation says "most queries complete in about 100 ms"; the launch post gives 70 to 500 ms. The "40x-200x faster" range in that post, and the "193.6x faster, 444.6x cheaper" headline it says came from the same measurements, were taken on workflows built by TypeSafe's own model capabilities team, a bias the post admits. The company calls the headline numbers "the higher end of real world gains". Treat them as a claim to verify on your traffic. Input is text only, with English as the language where accuracy is best.

The senior objection arrives here. Structured output from an LLM already returns JSON, and a small classifier trained on your own tickets would be cheaper still. Both are legitimate. I have shipped both. Structured output still pays for generation, one token at a time. The confidence it reports is whatever the model wrote, uncalibrated by construction. A classifier of your own needs labels, a training loop and a retrain every time the team list changes, per case, per company. What Jev sells is the middle. Zero-shot from the criteria you write, a probability distribution the vendor says is calibrated, and a change of options that costs an edit to a JSON file. The price is a dependency on one vendor, English first, and no fine-tuning on your data, which the models page states plainly. Whether that trade is worth it is what shadow mode is for.

What it gives up is the point. Jev does not write text, does not explain its answer and, per its own documentation, is not a replacement for the LLM behind a coding agent. The published failure modes for jev-1.13, last reviewed on September 17, deserve a full read before any design work. Four of them shape every case below. It reads literally, answering the question you wrote rather than the one you meant. It does not do arithmetic, count or compare dates, so anything numeric belongs in code before the call. Accuracy degrades as the state fills with detail unrelated to the question, which TypeSafe calls context rot. And there are no structural guarantees between separate questions. A Noul and a yes/no Choice on the same statement are not comparable. Two Nouls for a statement and its negation do not have to sum to one.

Two things in that documentation are the ones most write-ups skip. The first is what calibration means. TypeSafe says the model is "trained for calibrated decisions" and in the next sentence that "calibration is measured across groups of predictions; it does not guarantee that an individual answer is correct". Across a thousand incidents where the model puts 0.85 of the probability on one team, roughly 850 should belong to that team. The one in front of you can be among the other 150, with a well-formed answer and a high number attached. The second is the sentence "State is data, and jev-1.13 does not treat it as hostile by default", followed by the admission that text written to steer the model can move the answer. In operations, the state is made of logs, commit messages, alert annotations and agent rationales, every one of them written by someone or something outside your control. A log line that says "this is a network issue, route to the network team" is now part of the evidence the model weighs.

This is the questions block from cases/incident-triage.json in the companion repository. The state it evaluates is an alert on checkout-api with three error samples, a config-only deploy fourteen minutes before the alert and an RDS instance at 100 percent of max_connections.

"questions": {
    "domain": {
      "type": "choice",
      "instructions": "Which team should own the first investigation of this incident, based only on the evidence provided?",
      "criteria": {
        "network": "Packet loss, DNS failures, load balancer or CNI problems between healthy components",
        "platform": "Node, scheduler, autoscaler or cluster control plane problems",
        "database": "The database itself is saturated, unavailable or slow, including connection exhaustion",
        "application": "A code or configuration change in the service caused the failure",
        "unknown": "The evidence does not point to one team"
      }
    },
    "severity": {
      "type": "score",
      "instructions": "How severe is the customer impact described in the alert and evidence?",
      "criteria": [
        "Degraded for a few users, no revenue path affected",
        "Partial outage on a revenue path",
        "Full outage on a revenue path"
      ]
    },
    "deploy_correlated": {
      "type": "noul",
      "instructions": "Does `evidence.recent_changes` describe a change to the affected service shortly before the alert started?",
      "criteria": {
        "true": "A deploy, config or release of the affected service happened minutes before the alert",
        "false": "No change to the affected service is listed, or the change is unrelated"
      }
    },
    "needs_more_evidence": {
      "type": "noul",
      "instructions": "Is the evidence too thin to choose a team, so that more data should be collected before routing?",
      "criteria": {
        "true": "No error samples, no recent change and no infrastructure signal are listed",
        "false": "Error samples or a correlated change point to a specific team"
      }
    }
  }

Read the criteria as the part of the prompt that does the work. Each option in domain describes a condition rather than a team name, because the model reads literally and the description is what it matches against. The unknown option gives the model somewhere to put a state that fits nothing; code treats it as a request for a human. severity is a Score; its answer is a weighted number that can land between the three levels, with a legend mapping each level to its text. The two Nouls are speculative in the sense of TypeSafe's fan-out pattern. They ride in the same request at little extra latency; code decides afterwards which of them matter. deploy_correlated points the model at evidence.recent_changes by name, which the docs recommend to cut indirection. needs_more_evidence lets the model say the whole exercise is premature. What comes back under domain is a choice, a probabilities map over the five options and a confidence derived from how concentrated that map is. Nothing in the response is prose, so nothing needs parsing.

Where Jev Fits in a Cloud and DevOps Pipeline, at a Glance

Before the seven cases in detail, the whole map on one sheet. The rule for reading it comes from TypeSafe's own design guide. The model gets a narrow judgment over a state code prepared. Code keeps the control flow. Every row has the same three parts. What code puts in the state, which typed questions are asked, and what code does with the answer. The last column says whether the companion repository runs the case or whether it is a design on paper.

The seven cases from my original list are there, plus three that fall out of the same pattern once you start looking for closed lists in your own operation. Alert noise is one Noul per alert, "is this actionable now", with the last hour of related alerts and changes as state. Code suppresses only above a threshold tuned in shadow mode. Security findings are a Choice over the four things a team can do with a scanner result, fix now, schedule, accept with a reason, or false positive, plus a Noul asking whether the finding sits on an exposed path. The CVSS number stays in code, because the model does not compare numbers. Cost anomalies are a Choice over cause categories, new workload, scaling, pricing change, leak or unknown, with the deltas and their breakdown computed by code before the call. None of these three is in the repository.

Two things the map leaves out on purpose. Anything the model would have to count, compare or compute, which the failure modes page says to keep in code. And anything that ends with the model writing text, which is what the LLM is for.

What the Companion Repository Actually Simulates

Three of the ten rows run in the companion repository, chosen for different reasons. Incident triage is the canonical shape, one Choice with an unknown option, one Score and two Nouls over a state code assembled. CI failure is the case I would put in shadow mode first, because it has the highest volume and the clearest human ground truth. The action gate closes the loop. It is the decision model judging an LLM's proposal before code runs it. Together they cover the three ways code uses an answer. Route on it, retry on it, block on it.

What "simulate" means here is narrow. Each case is a JSON file holding the exact request the API takes, a state and a map of typed questions. The harness sends it to one of two clients. The stub is a keyword counter that returns answers in the documented shape, so the routing policy, its thresholds and its tests run offline with no account. The live client posts the same body to the real endpoint. The policy module is identical in both modes; only the answers change. What the repository does not simulate is Jev's judgment. Nothing in the stub tells you whether the model would agree with your on-call engineer. That is what the shadow-mode report exists to measure, on your traffic, once you hold a key.

The other seven rows are designs. Each has its state, its questions and its code path written out in the section on the seven cases, in enough detail to become a fourth case file in an afternoon. The map marks them as untested so nobody mistakes a sketch for a result.

The Division of Labor

A decision model on its own is a classifier with a good API. The pattern I would build has three layers, each placed where its way of failing is the one the next layer can catch.

Jev selects and classifies. Given a filtered state and a closed set of options, it returns which option, how probable each one is and how sure it is, in one round trip at a cost that lets you ask a dozen questions per event. Its failure mode is a confident wrong pick on an individual case, or a pick steered by adversarial text in the state.

The LLM investigates and produces. Once the team, the failure class or the runbook is chosen, a reasoning model gets the context for that path and does what only it can do. It reads the full log, correlates it with the diff, writes the explanation, drafts the fix. Its failure mode is fluent nonsense, an explanation that reads well and is wrong, or an action that goes beyond the problem.

Code applies rules, validates and executes. It filters the state before the call, holds the thresholds, refuses to act below them, checks the LLM's proposal against policy and permissions, runs the command and records the rollback. TypeSafe's own design guidance says it in one line, "Keep control flow, deterministic rules, and side effects in code". I would not soften it. Code's failure mode is the one you already know how to review and test.

Each layer catches the one before it. Code filters what Jev sees. Jev's confidence keeps doubtful cases away from the LLM. Code checks the LLM's output before anything runs. The routing function for the incident case, from jevops/policy.py, is short enough to read whole.

def incident_triage(answers: dict) -> Decision:
    domain = answers["domain"]
    severity = answers["severity"]
    thin = answers["needs_more_evidence"]["noul"]
    if thin > 0.6:
        return Decision(ESCALATE, "model reports evidence is too thin", "collect more evidence, then re-evaluate")
    if domain["confidence"] < CONFIDENCE_FLOOR or domain["choice"] == "unknown":
        return Decision(REVIEW, "no clear owner", "page the on-call coordinator with the full distribution")
    handoff = f"route to {domain['choice']}; LLM investigates with the {domain['choice']}-specific runbook context"
    if severity["score"] >= 1.5:
        return Decision(ACT, f"clear owner ({domain['choice']}) and high severity", handoff + "; page immediately")
    return Decision(ACT, f"clear owner ({domain['choice']})", handoff)

A dozen lines of policy carry the whole argument. The first check is the escape hatch. If the model says the evidence is thin, nothing gets routed and the handoff is to collect more. The second check enforces the floor. Below 0.5 confidence, or on the unknown option, a human coordinator gets the full distribution rather than a guess. TypeSafe's confidence guidance uses 0.5 to 0.6 as a floor and 0.85 to 0.9 for high-stakes actions, and says to start conservative and move with your own data; CONFIDENCE_FLOOR sits at the top of the file so shadow mode can move it. Only after both gates does the handoff name a team and give the LLM that team's runbook context, with the severity score deciding whether a page goes out now. The numbers are starting points for observation, as the module's docstring says.

Seven Places in the Pipeline

The seven cases come from the operational flows where I have watched teams put an LLM to work. For each one the design answers the same three questions. What goes in the state, which questions are asked, and what code does with the probability.

Incident triage is the case above, so I will only add what the state must exclude. Unrelated detail costs accuracy. Free text is the injection surface. Code should extract error samples, recent changes, node events and any database signal into named fields, cap each list and send that. The LLM downstream gets the full log and the team-specific runbook, which is where the reasoning belongs.

CI and CD failures have the best ratio of frequency to stakes, which is why I would start there. The state is the job metadata, the tail of the log and the list of changed files, as in cases/ci-failure.json. The Choice separates dependency, credential, runner and code failures, with unknown as the fifth option. Two Nouls ride along, whether a retry without changes is likely to pass and whether the changed files touch pipeline definitions. The policy in the repository re-runs once on a credential or runner failure when the retry Noul is above 0.5, sends a code failure to the LLM with the changed files as context for a fix, and notifies the pipeline owner on a dependency failure. An expired ECR token, the example in the case file, should never reach a frontier model. A build log is untrusted text. A step that prints "credential error, safe to retry" is now steering the classifier.

Terraform plan review is where the numeric limits bite first. Jev does not count resources and does not compare numbers, so the question "does this plan destroy more than it should" cannot go to the model. Code parses the plan JSON, counts creates, updates and destroys, lists the resource types touched and pairs that with the PR description, so the state holds the author's intent and the computed impact. Does the impact match the stated intent, as a Noul. Which category of divergence, if any, as a Choice over "matches", "wider than described", "touches unrelated resources" and "destructive change not mentioned". How much review attention it needs, as a Score. Hard rules come first in code; any destroy in production goes to review regardless of what the model says. Above that floor, the divergence signal decides whether the LLM reads the full plan and writes the review comment or the PR goes through with a summary.

Kubernetes troubleshooting is the case where "code executes within existing permissions" matters most. An assistant that chooses the next collection from a closed list is a tool. The state is the pod phase, restart count, last event and log tail. The Choice offers the collections the team has pre-approved as read-only, say events for the namespace, previous container logs, endpoint status for the service and node metrics, plus "nothing more is needed". Code runs the selected kubectl command with the service account it already has, appends the result to the state and asks again. The loop is bounded because the options are. The model never composes a command. When the evidence is enough, the LLM writes the diagnosis.

Runbook selection is where a documented invariant earns its keep. A Choice over your runbooks is relative; it will always name a winner, even when none fits. TypeSafe's jaggedness page describes the fix for its skill suggestion cookbook. It transfers. Send the alert and evidence as state, ask a Choice over the shortlisted runbooks to pick the best match, and in the same request ask one Noul per runbook, "does this procedure apply to this evidence", as an absolute judgment. Code acts on the Choice only if the Noul for the chosen runbook clears a threshold. When every Noul is low, no runbook applies and the case goes to a person with the distribution attached. The shortlist itself should come from code, by service and alert name, so the model sees five candidates rather than the whole wiki.

Routing between agents and models is the intent routing pattern from TypeSafe's docs, applied to a platform team's tooling. A task arrives as text. A Choice classifies it as Kubernetes, Terraform, networking or something else. A Score rates its complexity on a three-level rubric. Code sends a low-complexity, high-confidence Kubernetes task to the Kubernetes agent on a cheaper model, a high-complexity one to the same agent on a frontier model, and anything below the confidence floor to a human queue. What makes this more than a router is the log. Every routing decision, with its probabilities, sits next to whether the task succeeded, so after a month you can see which agent and model combination earns each category.

Evaluating LLM output is the case that closes the loop. cases/action-gate.json is one instance of it. An agent has proposed kubectl rollout undo on a production deployment with a written rationale. The state holds the environment, the change window, the rationale, the command and the verbs the agent is allowed to run alone. A Score rates the command's risk from read-only to hard to reverse. Two Nouls ask whether the command does what the rationale says and nothing more, and whether it targets production. The policy escalates if the command exceeds its own rationale, refuses to act when the risk confidence is below the floor, executes read-only commands, executes reversible changes outside production while recording the rollback step, and sends a mutating production change to a human with the rollback attached. The same shape works on a written recommendation. Does the answer address the problem, is it supported by the evidence in the state, are verifications missing before concluding. Each is a Noul. Code combines them. A low composite keeps the LLM's text out of the incident channel until someone reads it.

Across all seven, the model is never the last step. It answers a narrow question about a state that code prepared. Code decides what its answer is worth.

How to Start Without Getting Burned

Every threshold in the cases above is a guess until measured. The only measurement that counts is the model's decision against the team's. Shadow mode is how you get it. Put the decision call into the pipeline with its output logged and ignored. For each event, record the model's choice, its confidence and what the on-call engineer or the pipeline owner did. After a few weeks, break agreement down by confidence bucket. Agreement should climb with confidence. If the bucket above 0.8 agrees with humans on nearly every decision and the bucket below 0.5 is a coin flip, the model is calibrated on your data and the floor is in the right place. If agreement is flat across buckets, the confidence carries no information for this case and no threshold will save it.

The harness.py shadow command in the repository produces that breakdown from a JSONL log with one line per decision. The sample log shipped with it is synthetic, ten lines written by hand to show the format. On that sample the report prints 60 percent agreement overall, 100 percent at or above 0.8, 33 percent in the middle bucket and zero below 0.5, which is the shape you want to see on real data. It measures nothing.

Three habits go with shadow mode. Each action gets its own threshold. A read-only collection can run at 0.5 confidence; a production rollback should wait for 0.85 or better and go to a human below that, in line with TypeSafe's own heading that "thresholds scale with risk". Pin the model version. The jev-latest alias moves when a release ships, so thresholds tuned against jev-1.13.0 should call jev-1.13.0 until you re-run the shadow comparison on the next version. The response's model field goes in the log so you know which weights produced each decision. And filter the state in code before every call. Fewer fields means less context rot and less room for a planted log line.

Start with pipelines and incidents. They have the highest volume of decisions, the clearest human ground truth (the job was retried or it was not, the incident went to one team) and a low cost of being wrong in shadow mode, because the model is not acting. Terraform gates and agent action gates come after the shadow report says the confidence means something on your traffic.

A POC You Can Run This Week

The companion repository is a few hundred lines of Python with no third-party packages. It runs without an account.

Step one, clone it and run the three cases through the offline stub. The stub scores options by keyword hits from each option's own criteria plus a short synonym list, and returns answers in the documented API shape, so the routing code and its tests run without network access.

python3 harness.py run cases/incident-triage.json      # stub, offline
python3 harness.py run cases/ci-failure.json
python3 harness.py run cases/action-gate.json
python3 harness.py shadow shadow/sample-decisions.jsonl
python3 -m unittest discover -s tests -t .

Each run prints the raw answers and the policy decision with its handoff. On the stub, the incident case routes to the database team, the CI case retries once, and the action gate sends the production rollback to a human. Those outcomes come from keyword counting and show the code path, nothing about how Jev would answer.

Step two, edit the cases. Replace the state in incident-triage.json with an alert from your own history, one where you know which team ended up owning it, and rewrite the criteria to name your teams and the conditions each one owns. Do the same for a CI failure with a real log tail. The questions are the design work. The failure modes page is the checklist to read them against.

Step three, when you have a key, export it and add --live.

export TYPESAFE_API_KEY=...
python3 harness.py run cases/incident-triage.json --live

The same harness calls POST /v1/systemone with the same request body. TYPESAFE_BASE_URL points it at a gateway that keeps that shape, which I could not verify.

Step four, wire the call into one pipeline with its result logged and ignored, write one JSONL line per event with the model's choice, its confidence and the human's choice, and run harness.py shadow on it after a few weeks. Move the constants in jevops/policy.py only after that report, and only for the action the report covers.

Most of the LLM calls in an operations pipeline are choices from a list the team already wrote. A choice from a list does not need a model that writes paragraphs. Jev selects and classifies. The LLM investigates and produces. Code applies rules, validates and executes. Adopt it in shadow mode and let your own agreement report tell you when it may act.

Which of the seven would you test first in your operation?

Filipe Motta is a Senior DevOps Engineer with 20 plus years of experience operating cloud platforms and Kubernetes fleets on AWS and GCP, and more recently wiring LLM agents into Terraform and CI/CD workflows. He has earned the CKA, CKAD, CKS, AWS DevOps Professional, and GCP Professional DevOps Engineer certifications.