7 Best Local LLMs in 2026, Matched to Your Hardware | Zeniteq
Loading
Loading article
Zeniteq is now available on mobile for iPhone, iPad and Apple Watch.Download nowLearn more
7 Best Local LLMs in 2026, Matched to Your Hardware
Seven open-weight models compared by the memory they need, how well they code, and the license terms attached, from 8 GB laptops to 128 GB workstations.
This guide evaluates seven open-weight models to help developers and hobbyists match specific large language models to their available hardware, from 8 GB laptops to 128 GB workstations. By comparing Qwen3.8-27B, Devstral Small 2, Gemma 4 26B A4B, gpt-oss-20b, and Nemotron 3.5 Bolt E4B, users can determine which model balances memory footprint against coding performance. This analysis helps you decide whether to prioritize dense reasoning for complex agentic tasks, speed for real-time chat, or compact sizes for limited VRAM, ensuring your chosen model runs efficiently without spilling into system memory.
You've got a graphics card with 16 GB of memory, or a Mac with 32 GB, and you want the best local LLM that will actually fit on it. That question has a different answer every few gigabytes, and the fall releases made it messier: one new model is named after a memory size its own maker's recipes don't use.
We compared seven open-weight models using their official model cards and the download sizes Ollama lists for each build, as of November 2026. Every benchmark score here comes from the company that made the model, so read them as vendor claims, and we say which lab ran which test.
This guide is for developers and hobbyists running models on their own PC or Mac. We judged each model on four things: how much memory it really needs, how well it codes (so it doubles as a guide to the best local LLM for coding), how fast it runs on consumer hardware, and what its license lets you do. The order starts with the strongest picks for a 24 GB machine, then moves down and up the hardware ladder, so it isn't a strict ranking. If you'd rather pay per token than buy memory, our open-weight coding model comparison covers the hosted prices.
1. Qwen3.8-27B
Best for: Developers with a 24 GB GPU or a 32 GB Mac who want one model for coding, chat, and screenshots
Qwen3.8-27B is the dense 27-billion-parameter model from Alibaba's Qwen3.8 generation. "Dense" means every parameter works on every token, which makes it slower than the mixture-of-experts (MoE) models further down this list but steadier on long, multi-step jobs. It reads images and video natively, and it ships under the Apache 2.0 license, so you can build on it commercially without extra terms.
Key features that matter on your own hardware:
Size on disk: The default Ollama build is , with an 8-bit build at 30 GB
Coding is where it pulls ahead. On Qwen's own table, the 27B scores 61.7 on SWE-bench Pro, a test where the model fixes real GitHub issues, up from 53.5 for Qwen3.6-27B. Qwen ran that test inside the Claude Code agent harness, so it reflects agent-style work: reading files, running commands, and patching code. The same table lists 90.3 on LiveCodeBench v6 and 73.0 on Terminal-Bench 2.1.
That makes it the first model I'd point an agent at on a single RTX 3090 or 4090. Ask it to read a stack trace screenshot, find the bug, and patch two files, and every step stays on your machine.
The catch is memory headroom. The 18 GB file leaves about 6 GB on a 24 GB card for the context cache (the memory that grows as the conversation gets longer), so in practice you'll run a far shorter window than the card's maximum. On a 16 GB card it doesn't fit at 4-bit without spilling into system RAM, which slows it badly. Before you commit, check how much context your agent tool sends with each request; that number decides whether 24 GB is enough.
2. Devstral Small 2
Best for: Agentic coding in a terminal or IDE on one 24 GB card or a 32 GB Mac
Devstral Small 2 is Mistral's 24-billion-parameter model built for coding agents, the kind that explore a codebase and edit several files in one go. It's the December 2025 build (the 2512 in its name), and version 2 added image input on top of its predecessor. Mistral says a single RTX 4090 or a Mac with 32 GB of RAM is enough to run it.
Vision: Reads images such as a screenshot of a failing test
Agent tools: Works with Mistral's own Vibe CLI, plus Cline and Kilo Code
License: Apache 2.0, for commercial and non-commercial use
Mistral's numbers put it at 68.0% on SWE-bench Verified, level with the 355-billion-parameter GLM 4.6 in the same table. For a 15 GB download, that's a lot of repo-fixing skill per gigabyte. It also leaves you about 3 GB more room for context than Qwen3.8-27B on the same card.
It's the model to pick if your day is spent in an agent loop, such as Cline proposing a diff, you approving it, and the agent running the tests. A tight model focused on code fits that pattern well, and the vision support means you can paste in a UI bug without describing it in words.
The trade-offs show up outside the editor. Mistral's own table lists just 22.5% on Terminal Bench 2, well behind its big sibling Devstral 2, so long chains of shell commands aren't its strength. It's also close to a year old, and Mistral reports SWE-bench Verified while Qwen reports SWE-bench Pro, so you can't line the two scores up directly. Run it on a repo you know well before you trust it with a large refactor.
3. Gemma 4 26B A4B
Best for: 16 GB to 24 GB machines where quick replies count for more than peak coding scores
Gemma 4 is Google DeepMind's open model family, released under Apache 2.0 in five sizes from E2B up to 31B. The 26B A4B is a mixture-of-experts model: it holds 25.2 billion parameters but activates only 3.8 billion per token. A router picks a few small "expert" networks for each word, so it answers much faster than a dense model of the same size.
Google's table gives the 26B A4B 77.1% on LiveCodeBench v6, a Codeforces Elo of 1718, and 88.3% on AIME 2026 math problems. The default 16 GB build uses quantization-aware training (QAT), meaning Google trained the model to expect 4-bit weights, which usually costs less quality than squeezing it after training.
That speed suits a chat assistant on a 16 GB card or a 24 GB MacBook Pro, where you want answers in a second or two and the occasional screenshot read. It's also the friendliest family for mixed hardware, since the same prompts carry down to the 12B on a smaller laptop.
Coding is the soft spot. Its 77.1 on LiveCodeBench trails the 90.3 Qwen reports for Qwen3.8-27B (different labs ran those tests, so treat the gap as rough). NVIDIA's own comparison put it at 37.22 on Terminal-Bench 2.1, well behind the newer Nemotron model further down. And on a 16 GB card, the 16 GB build leaves almost nothing for context. If you have 24 GB, the dense 31B at 19 GB scores higher on Google's table (80.0% on LiveCodeBench), so compare the two before you settle.
4. gpt-oss-20b
Best for: 16 GB machines and anyone who wants OpenAI-style reasoning offline
gpt-oss-20b is the smaller of OpenAI's two open-weight models. It has 21 billion parameters with 3.6 billion active, and OpenAI trained it with MXFP4 quantization (a 4-bit format) so it runs within 16 GB of memory. It's Apache 2.0, and it's the easiest model here to get running, since OpenAI's own instructions come down to one Ollama command.
Reasoning effort: Low, medium, or high, so you can trade depth for speed
Agent tools: Native function calling, web browsing, and Python code execution
Visible reasoning: Full chain-of-thought access, handy for debugging prompts
Fine-tuning: OpenAI supports full parameter fine-tuning
It fits where the others are tight. On a 16 GB GPU or a 16 GB Mac, it leaves roughly 2 GB for context, which is enough for a coding chat or a short agent session. The adjustable reasoning effort is useful on slower hardware: set it to low for quick lookups and high when you want it to work through a tricky function.
There's a price for that small footprint. OpenAI says it was trained on its harmony response format and won't work correctly without it, so tools that skip the chat template will give garbled answers. It's text-only (the card lists text generation only), and its 128K context is half what most models here offer. Its big brother, gpt-oss-120b, needs a single 80 GB GPU, so there's no easy middle step up. Check that your runner applies the harmony template before you blame the model for bad output.
5. Nemotron 3.5 Bolt E4B 4GB
Best for: 8 GB laptops, mini PCs, and Jetson boards running a chat or voice assistant
Nemotron 3.5 Bolt E4B 4GB is NVIDIA's retune of Google's Gemma 4 E4B, published on Hugging Face on October 23. It's a small model with 8 billion total parameters (4 billion effective) shipped as a single 4-bit build, and NVIDIA says it's designed for low-memory, low-latency use, leaving headroom for other apps.
Precision: One Q4_K_M build, so there's no quantization choice to get wrong
Context window: Up to 64K tokens in and 64K out
Thinking control: On or off, plus a runtime thinking budget
Speed add-on: A companion DFlash drafter for speculative decoding (a tiny model guesses ahead and the main model checks its guesses)
Test hardware: NVIDIA lists Jetson Orin and Jetson Thor alongside desktop GPUs
NVIDIA's table compares 4-bit builds head to head, and instruction following is where Bolt shines: 64.00 on IFBench against 42.67 for the Gemma 4 E4B it came from, and 55.60 against 39.36 on MultiChallenge, a multi-turn conversation test. For a voice assistant or a game character that has to stick to its rules across a long chat, those are the scores that count.
The same table is honest about its limits. Qwen3.5 4B beats it on Tau2-Bench (61.74 against 52.32) and on the BFCL-v3 function-calling test (70.56 against 67.31), so for tool-heavy agents a small Qwen may serve you better. It's text-only, it isn't a coding model, and NVIDIA's summary still marks the llama.cpp recipe as pending. The license is NVIDIA's OpenMDW-1.1, which allows commercial use, layered on Gemma's Apache 2.0 terms. Read both before you ship it inside a product.
6. Nemotron 3.5 Bolt 74B-A9B 24GB
Best for: 32 GB GPUs, DGX Spark, and 64 GB-plus machines running local agents
Nemotron 3.5 Bolt 74B-A9B 24GB arrived on Hugging Face on October 15. NVIDIA shrank its 120-billion-parameter Nemotron 3.5 Super VL to a 74B model with 9B active parameters, then trained it to run at 2-bit precision, so the weights fit in 24 GB. It reads text, images, and video, and it handles up to 256K tokens of context. Our Nemotron 3.5 local setup guide walks through running the wider family on a DGX Spark.
32 GB recipe: For a 48 GB-plus GPU or a 128 GB Spark, with the full 262,144 tokens
Inputs: Text, images, and video, with text output
Design: A hybrid of Mamba-2 layers (a faster alternative to attention for long inputs) and Transformer layers, with mixture-of-experts routing
On NVIDIA's chart, Bolt leads the other local-size models on agent work. It scores 50.98 on Terminal-Bench 2.1, against 44.38 for Qwen3.6-35B-A3B and 37.22 for Gemma 4 26B A4B, and 66.75 on the BrowseComp web-research test against 48.74 for the Qwen. NVIDIA says English and programming languages are its primary focus, which fits a terminal agent that browses docs and runs commands for you.
The name is the trap. Despite the "24GB," NVIDIA's own recipes start at a 32 GB graphics card, because the model also needs room for its drafter and context cache, so an RTX 4090 owner isn't covered. The recipes cover NVIDIA hardware only, with no Mac instructions. And it doesn't win everything: the Qwen beats it on PinchBench (88.07 against 87.82) and on avoiding hallucinations (65.85 against 61.27). If you own a 32 GB card or a Spark, start with the 28 GB recipe and measure your real context needs before moving up.
7. Qwen3.8-Flash-Next
Best for: 128 GB Macs and workstations chasing top coding scores at home
Qwen3.8-Flash-Next is an experimental preview of the architecture Qwen plans to use for Qwen4. It has 125 billion parameters with only 6 billion active per token, plus 51 billion in a new n-gram embedding table, a big lookup table indexed by short word sequences. Qwen says that table needs less compute than adding experts and is easier to offload when memory is tight. Qwen's hosted Qwen3.8-Flash model is built on it.
Size on disk: The smallest Ollama builds are 105 GB (MLX and NVFP4), and Q4_K_M is 189 GB
Active parameters: 6 billion per token, so it generates fast for its size
Context window: 262,144 tokens natively, extendable to 1 million
Attention: A hybrid of Gated DeltaNet and Qwen Sparse Attention, which reads context in small blocks to cut long-context lag
License: The Qwen Community License 1.0, a custom license
Qwen's comparison table puts it ahead of its smaller sibling on agent coding: 62.5 on SWE-bench Pro against 61.7 for Qwen3.8-27B, 58.7 against 42.2 on DeepSWE 1.1, and 81.0 against 73.8 on SWE-bench Multilingual. If you write code in several languages and own a 128 GB Mac Studio or similar, that multilingual jump is the reason to try it.
The math is hard to love, though. Its SWE-bench Pro lead over the 27B is under a point, and it needs about six times the memory (105 GB against 18 GB). It's labeled experimental, and its license is a custom one, unlike the Apache 2.0 on Qwen3.8-27B, so read the terms before you build a product on it. Unless your work leans on the multilingual or long-horizon scores, the 27B on a cheaper machine gets you most of the way.
How to choose the right local LLM?
Choose the best local LLM by starting with your memory, then your main job (coding, chat, or agents), then the license. The model that tops a benchmark is useless if it spills out of your GPU and crawls through system RAM.
What to look for in a local LLM
Memory headroom: Add the download size to the context cache you'll need. If you size for the file alone, long prompts push the model into system RAM and speed drops sharply.
Active versus total parameters: MoE models like Gemma 4 26B A4B and gpt-oss-20b run at the speed of their active parameters. Ignore this and you'll pick a dense model that types too slowly for chat.
The context you'll actually use: A 256K maximum means little on a 24 GB card. Coding agents send whole files, so check their request size first.
License terms: Four models here are plain Apache 2.0. The two Nemotron models use OpenMDW-1.1, and Qwen3.8-Flash-Next uses a custom Qwen license. Skip this and you may have to swap models after you ship.
Runner support: Most of these run in Ollama, LM Studio, or llama.cpp, but gpt-oss needs its harmony template and the Nemotron recipes target llama.cpp on NVIDIA hardware.
Coding: Qwen3.8-27B and Qwen3.8-Flash-Next post the highest SWE-bench Pro scores here, while Devstral Small 2 is the most code-focused of the 24 GB options.
Speed: Gemma 4 26B A4B, gpt-oss-20b, and Qwen3.8-Flash-Next activate 6 billion parameters or fewer per token, so they feel quicker than the dense Qwen3.8-27B.
Agents and browsing: Nemotron 3.5 Bolt 74B leads NVIDIA's terminal and web-research comparisons, as long as you have 32 GB of GPU memory or more.
Vision: Qwen3.8-27B, Devstral Small 2, Gemma 4, Bolt 74B, and Flash-Next read images; gpt-oss-20b and Bolt E4B are text-only.
Licensing: The Apache 2.0 models carry the fewest strings, and the Nemotron and Flash-Next licenses need a read before commercial use.
What fits your machine
These pairings use each model's download size plus a few gigabytes for context:
8 GB laptop or Jetson board: Nemotron 3.5 Bolt E4B, or Gemma 4 E4B at 6.6 GB if you want image and audio input.
16 GB GPU or Mac: gpt-oss-20b at 14 GB leaves about 2 GB for context. Gemma 4 26B A4B at 16 GB technically fits, with almost nothing to spare.
24 GB GPU or 32 GB Mac: Qwen3.8-27B at 18 GB or Devstral Small 2 at 15 GB, with roughly 6 to 9 GB left for context.
32 GB GPU or 64 GB Spark: Nemotron 3.5 Bolt 74B with NVIDIA's 28 GB recipe and 65,536 tokens of context.
128 GB Mac Studio or Spark: Qwen3.8-Flash-Next at 105 GB, or Bolt 74B with its full-context recipe.
Which local LLM should you choose?
If you need one model for coding and screenshots on 24 GB, pick Qwen3.8-27B.
If you live in an agent loop inside your editor, pick Devstral Small 2.
If you want fast chat on a 16 GB card, pick gpt-oss-20b, or Gemma 4 26B A4B if you also want images.
If you're building a voice or device assistant on 8 GB, pick Nemotron 3.5 Bolt E4B.
If you have a 32 GB card or a DGX Spark and run browsing agents, pick Nemotron 3.5 Bolt 74B.
If you own a 128 GB machine and code in several languages, try Qwen3.8-Flash-Next.
Skip local models entirely if your machine has under 8 GB of free memory, or if you need the very top coding scores on huge repos every day. A hosted model will cost less than the hardware upgrade.
Final thoughts
The best local LLM in late 2026 is the one that fits your memory with room to spare, and for most developers that means a 24 GB card running Qwen3.8-27B or Devstral Small 2. The newer releases push both ends of the ladder, with NVIDIA's Bolt models covering 8 GB devices and 32 GB agent rigs, and Qwen3.8-Flash-Next showing what 128 GB buys.
My pick is Qwen3.8-27B, because it pairs the strongest coding scores you can fit on one consumer GPU with vision and a plain Apache 2.0 license. I'd only reach past it for Devstral Small 2 if my whole day ran through an editor agent, or for Bolt 74B if I already owned a 32 GB card.
What do you think of this list? Is there a local model you'd add, or one you think doesn't deserve its spot? Tell us which models are running on your machine right now.
What do you think of this list? Share your thoughts in the comments.
Frequently Asked Questions
5 questions
1
What is the best local LLM for coding?
The best local LLM for coding on a 24 GB machine is Qwen3.8-27B, which Qwen reports at 61.7 on SWE-bench Pro. If you mostly run a coding agent inside your editor, Devstral Small 2 is the leaner pick at 15 GB.
2
How much RAM do I need to run a local LLM?
You need about 8 GB for a small assistant model and 16 to 24 GB for a capable coding model. Plan for the download size plus several gigabytes for the context cache, since longer prompts use more memory.
3
Can I run a local LLM on a Mac?
Yes, Apple silicon Macs run local LLMs well because the GPU shares the Mac's unified memory. A 32 GB Mac handles Qwen3.8-27B or Devstral Small 2, and Ollama offers MLX builds made for Apple chips.
4
Are local LLMs free for commercial use?
Yes, for most models here, since Qwen3.8-27B, Devstral Small 2, Gemma 4, and gpt-oss-20b use Apache 2.0. The Nemotron models use OpenMDW-1.1 and Qwen3.8-Flash-Next uses a custom Qwen license, so read those terms first.
5
Do local LLMs work offline?
Yes, once the weights are downloaded, a local LLM runs with no internet connection. Your prompts and code never leave your machine, though any web-browsing tools you give an agent will still need a connection.
