The developer of Janas reports running Qwen3-Next-80B-A3B, a 48.4GB model file, on a 32GB laptop at 24 tokens per second without draft prediction. The documented llama.cpp configuration reached 2.8 tokens per second.
Janas keeps frequently used mixture-of-experts weights in memory and fetches other required experts from an NVMe SSD. The open-source C implementation, licensed GPL-3.0-or-later, is designed around that storage-to-memory path.
The result is worth investigating, though it isn't an independently established performance advantage across hardware. These developer-reported measurements center on one Intel laptop. The repository’s October 12, 2026 Windows update broadens the installation options without expanding the hardware evidence.
Janas Loads the Experts a Token Needs
In a mixture-of-experts, or MoE, model, a router selects experts to process each token. Janas uses that selective execution to avoid keeping every expert’s weights resident.
Developer Maurizio “camauri” Cammalleri describes the layout in his Hugging Face announcement. Janas converts GGUF files into its JNS format without re-quantizing them. Each expert gets an aligned storage slot containing its gate, up, and down weights together, allowing the engine to fetch that expert with one aligned request. Other model data occupies a resident region.
On Linux, the engine uses O_DIRECT and io_uring to read experts while bypassing the operating system’s page cache. Its own expert cache retains experts that recur. Cammalleri reports that this cache grows to roughly 19GB for the 80B model on his laptop, with uncached experts coming from storage.
Simply memory-mapping a model larger than RAM doesn't provide this arrangement. Janas organizes storage around the units the router selects and manages which units stay in memory. It can also apply the next layer’s router early and overlap expert reads with the current layer’s computation.
Cache misses still require disk access. A fast NVMe drive and useful expert reuse can make the design effective; a slower drive or different routing behavior can change the outcome. The reported 48.4GB figure describes the model file and offers no guarantee that every similarly sized model will run at the same speed.
The Useful Comparison Is 24 Versus 2.8, Not 37 Versus 2.8
The project’s attention-grabbing 37-token-per-second result includes drafts from Qwen3-Next’s own multi-token prediction block. The main model verifies those predictions before accepting them. According to the developer, that process preserves the text the model would otherwise generate.
For the comparison without drafts, the repository reports:
| Model | Janas prompt processing | llama.cpp prompt processing | Janas generation | llama.cpp generation |
|---|---|---|---|---|
| Qwen3-Next-80B-A3B, 48.4GB | 62 tok/s | 16 tok/s | 24 tok/s | 2.8 tok/s |
| Qwen3-4B | 193 tok/s | 164 tok/s | 27 tok/s | 18 tok/s |
These developer-run comparisons used 1,024-token prompts and were measured October 9–11, 2026. Prompt processing consumes input; generation produces the reply. They are separate workloads.
The llama.cpp configuration matters especially for the beyond-RAM model. Cammalleri says its usual build ran out of memory when the GPU participated. The working comparison used a build with repacking disabled through GGML_CPU_REPACK=OFF, plus --no-host 1 -lm mmap to memory-map the experts. Those settings made the comparison runnable, but they do not establish the best achievable llama.cpp configuration on every machine.
The repository also publishes a separate janas-bench table using 256-token prompts followed by 64 generated tokens. Its figures are means of three rounds, with alternating order intended to distribute warming and cache effects. That table reports 22 tokens per second without drafts for Qwen3-Next, compared with the comparison table’s 24. These are separate measurements and should not be merged into one benchmark.
Draft speed is particularly workload-dependent. The developer measures it on a short English answer about making bread and reports that another model, Qwen3.5-2B, gained almost nothing on Italian prose because far fewer drafts were accepted. A short, draft-assisted reply is not a reliable forecast for a long coding session.
The Laptop and Its Limits Are Part of the Result
The repository’s hardware disclosure identifies the primary Linux test system as:
- Intel Core Ultra 9 185H, with six performance cores, eight efficiency cores, two low-power cores, and integrated Arc graphics.
- 32GB DDR5 RAM.
- A 1TB NVMe SSD, reported at approximately 5.7GB/s reading.
- Debian 13, kernel 6.12.107, Mesa 26.1.6 Vulkan drivers, and GCC 14.2.
Janas chooses threads and GPU use through measurements on the host. That adaptive behavior is useful, though it cannot substitute for testing other hardware. The developer specifically flags CPUs without AVX-VNNI, AMD processors, discrete GPUs, slower disks, and different memory capacities as largely untested.
Windows measurements used the same laptop, running Windows 11 build 26300 with Intel’s graphics driver. They add operating-system coverage without providing an independent hardware replication.
There are stability risks, too. Janas can consume enough memory to push the desktop into swap and make it slow or unresponsive. The developer also reports a GPU-driver reset that took the desktop with it. The public API and command line can change without notice.
These limits determine whether Janas is appropriate for a spare testing session or an always-on development service.
A Small Cache Can Change the Computation
Some Janas optimizations change the work the model performs.
Cammalleri documents a cache-conditional routing optimization that activates when the cache holds less than one-fifth of the experts. An uncached expert with a small routing weight can be skipped, with its contribution redistributed to the token’s other experts.
For the 80B model with a 4GiB cache, he reports an increase from 20.7 to 27.7 tokens per second. Agreement with llama.cpp’s most likely token falls from 393 to 384 positions out of 400.
That token-agreement check neither proves substantial degradation nor establishes unchanged quality; it is not a task-quality benchmark. A low-memory speed result needs its routing settings attached.
Verified drafting and read-ahead work differently. Drafting checks proposed tokens, and read-ahead changes when weights arrive. Skipping an expert changes which computations are performed.
Start With a Small Model and the GPU Disabled
The fastest cautious route is the prebuilt package, followed by a small-model smoke test. Don't begin by downloading the largest model and letting the engine consume all available headroom.
Sources
- Janasgithub.com
- Hugging Face announcementdiscuss.huggingface.co
