KV1-1.2B Is Trained for a 2-Bit KV Cache, and the Memory Math Is Striking

Sanket Chaukiyal

November 16, 2026

TL;DR

  • Klein ML released KV1-1.2B, an open model trained from the start to run with a 2-bit KV cache. It is built on Liquid AI’s LFM2.5-1.2B-Instruct.
  • At 32K context the cache shrinks from 384 MiB in fp16 to 60 MiB, a 6.4x cut, while the model keeps 75.8% of full-precision quality.
  • Against an untrained 2-bit baseline, it jumps from 43.1% to 77.6% on IFEval and from 1.6% to 44.8% on GSM8K. All figures are self-reported.
  • Knowledge-heavy tests suffer, and Hugging Face Transformers won’t deliver the savings without the team’s KV1.cpp fork.

A 60 MiB cache at 32K context

Klein ML has released KV1-1.2B, the first model in its KV1 family. The pitch is narrow and specific: the model was trained to expect a 2-bit KV cache, rather than having its cache squeezed after the fact. According to the Klein ML post on Hugging Face, “at a full 32K context, KV1-1.2B keeps its entire cache in 60 MiB, 6.4× less than the 384 MiB the same model needs in fp16.”

The scores tell the rest of the story. Squeezing the base model’s cache to 2 bits without training breaks it: LFM2.5-1.2B falls to 1.6% on GSM8K and 0.6% on HumanEval. KV1-1.2B scores 44.8% on GSM8K and 40.2% on HumanEval, plus 77.6% on IFEval against 43.1% for the untrained 2-bit version.

Overall, the model retains 75.8% of the full-precision original’s quality. It ships under the LFM Open License v1.0.

The training recipe is modest. Klein ML used 750M tokens at 32K context on a single RTX Pro 6000, roughly 17 GPU hours. It combined self-distillation against the frozen fp16 teacher, Hadamard rotations to spread outlier key channels across dimensions, attention sinks that keep the first 4 tokens in fp16, and straight-through estimator training so gradients can flow through rounded 2-bit values.

Teaching the model to live in the small bag

Quantizing a cache after training is like vacuum-sealing a loaf of bread that is already baked. You get a smaller package and a flattened loaf. KV1 bakes the loaf knowing it will spend its life in the bag. That is the whole idea, and the GSM8K jump from 1.6% to 44.8% suggests it does something real.

I have read plenty of quantization write-ups that promise a free lunch. This one at least shows the bill, which is why I am paying attention.

Now the competitive picture. At 32K context, the source lists cache sizes of 139 MiB for Gemma 3 1B, 1,024 MiB for Llama 3.2 1B and 3,584 MiB for Qwen3-1.7B. KV1-1.2B’s 60 MiB undercuts all three. On IFEval it also scores 77.6% against Qwen3-1.7B’s 68.2%, which is a bigger model by the numbers in the name.

But read the fine print. An untrained 3-bit cache sits at 84 MiB and still maintains higher overall quality than KV1-1.2B’s 2-bit setup. So is 2-bit training a better trade than simply using 3 bits? For some deployments, 24 MiB less per 32K sequence matters a lot. For others it will not.

The weak spots are plain. MMLU-Pro dropped to 21.1%, recovering only 54.6% of fp16 performance. GPQA-Diamond landed at 24.2%, which is chance level. Whatever 2 bits can preserve, it is better at following instructions and working through arithmetic than at recalling knowledge.

And there is a practical catch. Run the model in Hugging Face Transformers and the cache is stored in bf16, so you do not see the memory savings. You need Klein ML’s KV1.cpp fork. Until that path is easy, the 60 MiB figure is a lab number for most people.

Why the KV cache eats memory

The KV cache grows linearly with context length and batch size. On edge devices and long-context workloads, it can exhaust GPU memory before the model weights do. That is why shrinking it matters more than the headline parameter count suggests.

The standard fix, aggressive cache quantization, has a bad track record when applied to models that were not trained for it. Reasoning tends to collapse, which is exactly what the 1.6% GSM8K figure for the untrained baseline shows. Training around the damage is the alternative bet.

The practical audience is anyone running many concurrent long sequences or squeezing a model onto small hardware. A 6.4x smaller cache means more sessions per GPU or longer contexts on the same device. That is the logic, though the source does not report throughput or latency numbers, so we cannot say how it plays out in speed.

Also keep the base in mind. KV1-1.2B is a fine-tune of Liquid AI’s LFM2.5-1.2B-Instruct, not a model built from scratch. The cheap training run, about 17 GPU hours on one card, hints that the method may travel to other models. The source does not claim that, though, so treat it as a hypothesis.

Three checks before trusting the 2-bit claim

First, independent evaluation. Every number here comes from Klein ML’s own post. Watch for third parties reproducing the IFEval, GSM8K and HumanEval results, and for anyone testing the knowledge-heavy tasks where the model visibly struggles.

Second, tooling. The memory savings depend on the KV1.cpp fork today. If support lands in mainstream runtimes, the 60 MiB figure becomes something you can actually use. If it does not, adoption stalls.

Third, the rest of the KV1 family. This is the first model in the series, and the post calls it that. Whether the recipe holds up on other sizes and architectures will say more about the method than this one checkpoint can.

Editor's Note

What caught me is the training bill: about 17 GPU hours on a single card. If the method really holds, this isn't a lab-only trick. I'm less convinced by the headline than by the 3-bit comparison, because an untrained 3-bit cache at 84 MiB still scores better overall. That's the number I'd argue with. I want independent runs before I believe any of it.

– Sanket Chaukiyal, founder, SmartChunks

FAQ

What is KV1-1.2B?

It is an open model from Klein ML, built on Liquid AI's LFM2.5-1.2B-Instruct and trained to run with a 2-bit KV cache. It is the first model in the KV1 family and is released under the LFM Open License v1.0.

How much memory does it save?

At 32K context, the KV cache needs 60 MiB versus 384 MiB in fp16, a 6.4x reduction. That applies to the cache only, not the model weights.

How good is the model with a 2-bit cache?

It retains 75.8% of full-precision quality overall. It scores 77.6% on IFEval, 44.8% on GSM8K and 40.2% on HumanEval, but only 21.1% on MMLU-Pro and 24.2% on GPQA-Diamond. All results are self-reported.

Can I get the memory savings in Hugging Face Transformers?

Not directly. Transformers stores the cache in bf16, so realizing the savings requires Klein ML's KV1.cpp fork.

Sanket Chaukiyal — Editor at Smart Chunks

Sanket Chaukiyal

Technology editor • 12+ years in editorial

Sanket is the founder and editor of Smart Chunks. He spent over six years at Autocar India (Haymarket SAC Publishing) as Sub Editor and Senior Copy Editor, and later served as Account Director (Content) at Rite Knowledge Labs. He holds a Master's in Media and Communication from the Symbiosis Institute of Media and Communication.

All articles → • LinkedIn