llmfit is a Rust command-line tool that works out which open-source large language models your own hardware can comfortably run. It inspects the machine in front of you — CPU, system RAM, GPU or GPUs, VRAM and accelerator configuration — and then ranks hundreds of models, across popular quantizations, by how well they fit. It is for people who run models locally: developers experimenting with on-device inference, machine-learning engineers who want to know what a given box can hold, and anyone tired of downloading multi-gigabyte weights only to watch them crawl.
What it does
The core job is a fit check that happens before the download, not after it. You run one command, llmfit reads your hardware, and it produces a ranked list of models your machine can handle.
- Scans CPU, system RAM, GPU or GPUs, VRAM and accelerator setup.
- Scores hundreds of models and providers for quality, speed and fit.
- Presents the ranking in a terminal UI rather than a wall of text.
- Covers models across popular quantizations, so the same family appears at several memory footprints.
- Reports compatibility with the common local runtimes: Ollama, llama.cpp, MLX and LM Studio.
There is also a benchmarking side. From the terminal UI you can download a model, serve it and measure real tokens per second on your own machine. Every run is saved locally first, and you can contribute the results back to the project as a pull request straight from the interface — no gh CLI and no third-party account required. The point of that loop is stated plainly in the README: real numbers from real machines make the estimates better for everyone.
How it works
The recommendation rests on two things the tool can actually determine: what your hardware offers, and what a given model at a given quantization demands. Quantization is the lever that matters most here. A model that will not fit in VRAM at full precision often fits several steps down, with a cost in quality, and llmfit's job is to lay that trade-off out per model rather than leave you to do the arithmetic against a spec sheet.
Because the tool reasons about formats and runtimes rather than wrapping a single one, its output spans the local ecosystem — GGUF-style builds for llama.cpp and Ollama, MLX builds on Apple silicon, and what LM Studio can load. It is written in Rust and ships as a single binary; the project also signs its builds through SignPath.
Getting started
llmfit is published on crates.io as the llmfit crate and the repository is MIT licensed. Once installed, the usage is what the video describes: type llmfit in your terminal, with no configuration to write, and read the ranked list. From there you can stay in the terminal UI to pull a model down and benchmark it, or take the model name to whichever runtime you already use.
When to use it / when not
Reach for it when you are about to pick a local model and you are not certain what your hardware will tolerate: a new laptop, a workstation with an unfamiliar GPU, or a machine where you have only ever run one small model and suspect it could do more. It is also useful as a sanity check when a model you already run feels slower than it should, since the benchmark can put a number on it.
It is not an inference engine. llmfit does not replace Ollama, llama.cpp, MLX or LM Studio — it tells you what to feed them, and you still need one of them to serve the model. It does not train, fine-tune or quantize anything. And its scores are estimates until you benchmark: the project's own emphasis on collecting real measurements is an admission that predicted throughput and measured throughput are not the same number.
Alternatives
The usual alternative is guesswork — reading a model card, eyeballing a parameter count against your VRAM, downloading a few candidates and keeping whichever one does not stall. That works, slowly, and it costs bandwidth and disk on every wrong guess. Some runtimes will refuse or offload a model that does not fit, which tells you the answer only once you have already paid for the download. llmfit's contribution is moving that verdict to the front of the process and covering many models at once instead of one at a time.
If you run open-source models on your own machines, this is worth ten minutes. It solves a small, specific, genuinely annoying problem — the gap between a model's name and whether your box can run it — and it solves it without asking you to configure anything or adopt a new runtime. Its 35,000-plus stars in a few months suggest plenty of people recognised the problem immediately. The honest caveat is scope: it advises, it does not serve, and its estimates are only as good as the hardware data behind them, which is precisely why the benchmark-and-share loop exists. Treat it as the first command you run before a download, not as part of your serving stack.