llmfit Guide: Which Local LLM Fits Your Hardware?

llmfit answers the question that should come before every local-model download: what will run well on this machine? The free terminal tool detects your RAM, CPU, GPU and VRAM, then ranks hundreds of models by memory fit, estimated speed, quality and usable context. It also selects the highest-quality quantization it expects to fit.

That can save a 20 GB download and a wasted afternoon. It cannot prove that a model is accurate enough, licensed for your product or fast under production load. Use llmfit to make a shortlist, then benchmark the shortlist with your runtime and workload.

llmfit in 60 seconds

Install and get a recommendation before downloading a model

The lowest-friction macOS or Linux install is Homebrew. Windows users can install through Scoop. If you already use uv, uvx runs llmfit without a persistent install.

# macOS or Linux
brew install llmfit

# Windows
scoop install llmfit

# Run once through uv
uvx llmfit

Launching llmfit opens the interactive terminal interface. For a fast, reproducible buying workflow, the CLI commands are more useful:

# Confirm what llmfit detected
llmfit system

# Return five coding recommendations as JSON
llmfit recommend --json --use-case coding --limit 5

# Inspect assumptions for one candidate
llmfit info "MODEL NAME"

# Estimate hardware for a specific model and 8K context
llmfit plan "MODEL NAME" --context 8192 --json

The official CLI and automation reference also exposes hardware overrides, context caps and a local REST API. That makes the tool useful for scripts and node scheduling, not just one-off desktop selection.

How llmfit decides what fits

llmfit first identifies the acceleration backend, such as CUDA, Metal, ROCm, SYCL or CPU. It then evaluates every catalog entry against available memory and a quantization hierarchy from Q8_0 down to Q2_K. The tool tries to keep the highest-quality quantization that fits. If full context does not fit, it can retry with a shorter context.

The combined score changes by use case. Coding can favor a smaller specialist over a larger general model, while reasoning gives quality more weight and chat gives speed more weight. This is more useful than sorting by parameter count, but it is still a heuristic. Your acceptance set remains the final judge.

Why MoE support matters, and what it does not mean

Mixture-of-Experts models publish a large total parameter count but activate only part of the network for each token. A dense-only calculator can therefore produce a misleading compute estimate. llmfit reads MoE metadata, accounts for active parameters and models a path where active experts use VRAM while inactive experts are offloaded.

Do not interpret that as “only active parameters need memory.” The full weight set still has to exist somewhere in the serving path, usually system RAM, storage or both. Expert switching, CPU offload and PCIe traffic also affect speed. llmfit's published scoring documentation is a better starting point than dense parameter arithmetic, but a real runtime benchmark remains mandatory.

Quantization: the best fit is not always the best product

Quantization reduces memory by storing model weights at lower precision. llmfit walks down its supported quality hierarchy until a configuration fits. That is exactly what you want for exploration. For production, add two boundaries:

• Set the real context cap. A model that fits at 4K context may spill into RAM at 32K because the KV cache grows with context and concurrency.
• Set a quality floor. A Q2 model that loads is not automatically good enough for code generation, multilingual support or structured output.

Use llmfit plan with the context you intend to serve. Then compare at least two adjacent quantizations on a frozen evaluation set. If the higher-quality option barely misses the machine, the correct decision may be different hardware or an API, not more compression.

Where hardware detection is strong and where to verify it

The official platform matrix documents manual --memory, --ram and --cpu-cores overrides. An override changes the recommendation calculation. It does not add runtime support that the operating system or inference engine lacks.

Five checks between a green fit and a production decision

1. Confirm the detected machine. Save llmfit system --json with the decision record. Fix missing or shared-memory values before comparing models.
2. Filter by the real job. Use coding, reasoning, chat, multimodal or embedding filters. Do not let a general score choose a specialist workload.
3. Freeze context and concurrency. Model the input size and simultaneous requests you expect, not a one-user demo.
4. Benchmark the actual runtime. Measure tokens per second, time to first token, prompt processing, memory high-water mark and failure behavior. llmfit can benchmark a served model and save local results.
5. Review the whole deployment. Check model license, data handling, observability, upgrades, fallback, security and total cost.

A model can fit perfectly and still be the wrong commercial choice. Compare the engineering and idle-capacity cost with API spend using our local models versus APIs break-even framework. If you want a concrete consumer-hardware example, the Colibri GLM 5.2 hardware review shows how model size, memory bandwidth and workload change the answer.

When llmfit is enough, and when you need engineering work

The clean commercial path is staged. Use llmfit to eliminate obviously bad candidates. Spend benchmark time only on the top two or three. Move to self-hosting only when measured quality, latency, risk and total cost beat the managed alternative.

Limits you should write into the decision record

• Speed is estimated from memory bandwidth, model size and efficiency factors. Runtime kernels and workload shape can move the result.
• The catalog is embedded in the binary. Upgrade llmfit to receive model database updates.
• Use-case quality relies on curated family benchmarks and fallbacks, not your private evaluation set.
• Aggregated multi-GPU VRAM does not make different cards, interconnects and memory bandwidth identical.
• A model's ability to load says nothing about its license, prompt safety, domain accuracy or operational readiness.

These are not reasons to skip llmfit. They are the boundary that makes the tool useful. It is a fast pre-download filter and planning aid, not a substitute for inference engineering.

Sources and claim boundaries

Installation, commands, supported runtimes, privacy behavior and the MIT license come from the llmfit repository. Hardware detection, quantization, MoE handling, scoring and speed formulas come from the project's how-it-works documentation. Platform limits come from the platform support reference. The workflow for measuring tokens per second and time to first token comes from the benchmarking guide. Facts were checked on 27 July 2026. Wavect did not independently benchmark llmfit's estimates for this article.