The first two Jev videos were about what the thing is. This one builds something with it — a model router you run on your own machine.
The setup is a Next.js UI talking to a local FastAPI server. Every prompt goes to Jev first, and one call asks three questions at once against the same state:
- choice — what kind of task is this? Chit-chat, simple question, rewrite, code, reasoning, image.
- score — how much model capability does a good answer actually need? Trivial ("a 2B model handles this") through to frontier.
- noul — does this contain personal, client, financial, medical or credential data?
Because all three evaluate in parallel, asking three costs about the same time as asking one. What comes back is just numbers, so the routing is ordinary if/else. Chit-chat goes to MiniCPM5 2B running locally. Code goes out to DeepSeek V4.1 Flash on OpenRouter. An image request goes to Qwen Image 2.1, also local. Privacy above 0.5 and the whole thing stays on the machine regardless of everything else. Hard work escalates a tier — Opus 5 if it's switched on. And when confidence comes back low, the router doesn't guess, it falls back to a safe default.
The numbers are the interesting part. A "how's it going?" prompt was judged in 331ms, difficulty zero, routed local. Drop in a fake API key and privacy comes back at 0.96 and it never leaves the machine. Over a session of 53 requests, 75% were answered locally — and a full run of demos cost under one cent.
There's an honest flaw he flags before building: if you send the prompt to TypeSafe to ask whether it's private, you've already leaked it. So the second half swaps Jev out for SemIf running locally. Same decisions, same shape, 88ms, nothing leaves the box.
That's the real takeaway. Once the decision layer costs almost nothing and answers in under 100ms, you can rethink how agentic systems are structured — put a fast gate in front of everything and only spend real money when the gate says you have to.
No comments:
Post a Comment