Kev

Self-hostable 0.8B to 27B decision models for teams that need classifications, not prose

No video published yet. The write-up below covers what the tool does and how to try it.

Category
AI models & inference
Audience
Data & ML

Published Updated

Kev is an open-source family of small "decision" language models — 0.8B, 4B, 9B and 27B parameters — that answer structured questions about a text instead of writing prose. It is built by Jared Palmer, co-creator of the Vercel AI SDK and now an SVP at GitHub/Microsoft, on top of Alibaba's Qwen3.5 and Qwen3.8 base models. It is aimed at backend and machine-learning engineers who already push text through a classifier in production — ticket routing, moderation, tagging — and pay a hosted API for every one of those calls.

What it does

You hand Kev a text and a question, and it returns an answer plus calibrated probabilities for each option in a single pass. The README names three question types: yes/no (noul), multiple-choice (choice) and a rating (score). The target is pipeline work, where the next step in your code needs a label and a confidence number.

Kev is presented as an open, self-hostable clone of Jev, the closed hosted model behind a commercial product called TypeSafe's "System One" API. Kev's HTTP API and even its Python client are deliberately wire-compatible with TypeSafe's SDK, so a team can repoint existing Jev-calling code at a self-hosted Kev server and leave the rest of the application alone.

The headline figure comes from the project's own benchmark as reported in the README: on held-out "new source" data, Kev-27B scores 0.851 accuracy against Jev's 0.857 — within one point of the closed commercial model. That is the authors' claim on their own evaluation, not an independent measurement, and this channel does not run the tools it covers.

Four sizes ship together as "Kev 1.0": 0.8B (runs on a laptop), 4B (the recommended default), 9B, and 27B (needs an 80GB GPU).

How it works

The models are Qwen base models adapted to emit a distribution over a fixed answer set rather than tokens of text, which is why one forward pass is enough and why the output carries probabilities. The speed numbers are also the project's own: Kev-4B answers six questions about a short text in 18.1 ms of model time on an H100, and a single container serves about 101 requests per second.

Adapting the models to your own labels is part of the pitch. The README says a fine-tuning run that moves Kev-4B onto a company's labeled data costs about one dollar on a rented H100 through Modal, and that the run can be started by telling a coding agent "fine-tune Kev on my support tickets" using a packaged skill installed with npx skills add jaredpalmer/kev@kev-finetune.

Getting started

The repository is github.com/jaredpalmer/kev, and it is a normal clone-and-install project plus model weights — no release binary, app store build or sideloader involved. Because you host it yourself there is no API key and no per-call fee, but somebody still pays for the compute: the intelligence runs on your own machine or on a GPU you rent. The 0.8B model is described as laptop-sized, 4B is the default recommendation, and 27B requires an 80GB GPU. The roughly one-dollar fine-tune is Modal's bill, not the project's.

Note what the material does not say. The README quoted here lists no operating system, CPU, RAM or VRAM figure for the 4B and 9B models, and names no license, so budget for a Linux GPU box and check those details in the repository first. The only exact command in the material is the skill install above; for the server itself, follow the README.

When to use it / when not

The main catch: the model that comes within a point of Jev is the 27B, and that one needs an 80GB GPU. The size you can run on a laptop is the 0.8B, and the material gives no accuracy figure for it or for the mid-sized models, so the published quality claim and the cheap deployment are not the same configuration.

Use it instead of TypeSafe's System One API when per-call pricing starts to hurt at volume, when the text cannot leave your infrastructure, or when you want a model tuned on your own labels. Don't use it if you classify a few thousand items a month, have no GPU to spare, or are content asking a general model like Claude with a JSON schema — that is far less to operate, even if it costs more per call and gives you no calibrated probabilities.

Alternatives

The direct comparison is Jev itself: the same wire protocol, fully hosted, 0.857 on the project's benchmark, paid per call. A general-purpose chat model with structured output is the other thing most teams already have in place. A classifier you train yourself stays the cheapest to serve and the most labeling work. The material does not benchmark Kev against anything other than Jev.

Take Kev seriously if you run text classification at a volume where a hosted decision API is a real line item, or in a setting where sending customer text to a vendor is a problem. The wire compatibility makes migration a configuration change rather than a rewrite, and a roughly one-dollar fine-tune makes it cheap to find out whether a 4B model handles your labels. If you classify occasionally, or have no GPU budget, the hosted API or a general model you already pay for will stay the less painful choice.

More in AI models & inference

All of AI models & inference →