Putting Jev to work: a router that decides where your prompt goes

 The first two Jev videos were about what the thing is. This one builds something with it — a model router you run on your own machine.

The setup is a Next.js UI talking to a local FastAPI server. Every prompt goes to Jev first, and one call asks three questions at once against the same state:

• choice — what kind of task is this? Chit-chat, simple question, rewrite, code, reasoning, image.
• score — how much model capability does a good answer actually need? Trivial ("a 2B model handles this") through to frontier.
• noul — does this contain personal, client, financial, medical or credential data?

Because all three evaluate in parallel, asking three costs about the same time as asking one. What comes back is just numbers, so the routing is ordinary if/else. Chit-chat goes to MiniCPM5 2B running locally. Code goes out to DeepSeek V4.1 Flash on OpenRouter. An image request goes to Qwen Image 2.1, also local. Privacy above 0.5 and the whole thing stays on the machine regardless of everything else. Hard work escalates a tier — Opus 5 if it's switched on. And when confidence comes back low, the router doesn't guess, it falls back to a safe default.

The numbers are the interesting part. A "how's it going?" prompt was judged in 331ms, difficulty zero, routed local. Drop in a fake API key and privacy comes back at 0.96 and it never leaves the machine. Over a session of 53 requests, 75% were answered locally — and a full run of demos cost under one cent.

There's an honest flaw he flags before building: if you send the prompt to TypeSafe to ask whether it's private, you've already leaked it. So the second half swaps Jev out for SemIf running locally. Same decisions, same shape, 88ms, nothing leaves the box.

That's the real takeaway. Once the decision layer costs almost nothing and answers in under 100ms, you can rethink how agentic systems are structured — put a fast gate in front of everything and only spend real money when the gate says you have to.

SemIf — https://github.com/TheoLeeCJ/SemIf

Bespoke Nimble 9B — https://huggingface.co/bespokelabs/Bespoke-Nimble-9B

Decider — https://github.com/Mapika/decider

Alex Wortega's OpenJev — https://huggingface.co/AlexWortega/openjev

DiffusionGemma (vLLM PR) — https://github.com/vllm-project/vllm/pull/57250

NanoJev — https://github.com/TianyuCodings/NanoJev

Layla — https://github.com/NandhaKishorM/laya

Jev, and the week everyone cloned it

 For two years every frontier lab has been pushing in the same direction: reasoning. Longer chains of thought, bigger thinking budgets, models that will happily sit there for minutes before answering — and won't show you the chain of thought you paid for.

That's System 2 thinking, in Kahneman's terms. Slow, deliberate, effortful. But most decisions that actually sit inside software aren't System 2 problems. What kind of support ticket is this? Is this urgent? Did the agent's output break a rule? Those are gut calls. You shouldn't need 30 seconds for them, let alone two minutes.

A lab called TypeSafe AI just came out of stealth going the opposite way, with System One models and a model called Jev. You can't chat with it at all. It doesn't generate text. The founder, Diogo Almeida, was a lead author on the InstructGPT paper that made models good at talking to people, and his argument now is that chat is the wrong interface for software. Software doesn't want a paragraph. It wants a value it can use.

So you pass in a state — a ticket, a log, an agent trace — plus typed questions. Only three kinds: choice (pick one of these options), score (rate on my scale), and noul (yes/no, with a probability). What comes back is typed answers with real probabilities, in 70–500ms. Output tokens are free; input is four cents per million. Think of it as an if statement that understands language.

The honest caveats: there's no paper and no architecture diagram — just three hints (new architecture, parallel sampler, and RLCD, reinforcement learning for calibrated decisions). "Can't hallucinate" really means it can't break your schema. It can still be wrong.

Then the interesting part. Within 24 hours of that video, open replications started appearing — now 20 to 30 of them, and some are close. On JevBench, Jev scores 75.3 and SemIf — a frozen Qwen 3.5 4B with no training at all — sits right behind it. Everyone converged on the same trick: read the logits over the answer tokens and softmax, no sampling.

The gap is still real on hard, multi-hop work. The move there is a cascade — fast model first, escalate only when confidence drops.