Tencent’s Hy4 Flash Packs 220B Parameters but Wakes Only 12B per Token

Sanket Chaukiyal

November 18, 2026

TL;DR

  • Tencent-Hunyuan’s repository describes Hy4 flash as its most cost-effective Hy model: 220B total parameters, 12B active per token.
  • It takes text and image inputs and supports a context window of up to 1M tokens.
  • The repo reports a WorkIndex score of 42.1 at 1/13 the cost per task of Hy4 preview and 1/20 of Hy4.
  • Those figures come from Tencent’s own documentation, so treat them as self-reported until others reproduce them.

A 220B model that only wakes 12B at a time

Tencent-Hunyuan has published Hy4 flash in a GitHub repository, and the pitch is right there in the first line. “Hy4 flash is our most cost-effective Hy model to date, featuring 220B total parameters (including 64B Engram parameters) with just 12B active parameters per token.”

The parameter accounting is unusually detailed. The language backbone holds 155.5B, Engram takes 63.5B, the vision encoder (ViT) adds 0.43B, and a built-in drafter called DFly2 adds 0.85B. Active parameters per token come to 11.3B for text plus 0.43B for vision.

The backbone runs 42 layers: one dense feed-forward layer and 41 mixture-of-experts layers, linked by identity Hyper-Connections (iHC). Each MoE layer holds 512 routed experts and activates 16 per token. Those experts work in a lower-dimensional latent space, which the repo calls LatentMoE.

The drafter is a 5-layer module for speculative decoding. The repo lists a decoding speed of 150 tokens per second, though the material I have doesn’t say what hardware or setup sits behind that figure. Attention is hybrid, mixing sparse and sliding-window attention.

The 12B number is the headline, and the cost claim is the real test

Total parameter counts have become a vanity metric. What you pay for at inference is closer to what fires per token, and 12B active out of 220B is a steep ratio. Picture a workshop with 512 labelled drawers of fasteners per bench. A good carpenter opens 16 for the job at hand, and a pegboard of pre-sorted common combinations covers the routine stuff. That’s roughly the bet here: huge storage, tiny working set.

Does the bet pay off? Tencent says so. The repo reports a WorkIndex score of 42.1 at 1/13 the cost per task of Hy4 preview and 1/20 of Hy4. Those are comparisons against Tencent’s own models, which makes them useful but not neutral. Nobody outside Tencent has shown me how WorkIndex is built or what the cost basis includes.

The competitive framing is where it gets interesting. The base-model benchmark table sets Hy4 flash Base against Hy4 Base, Hy3 preview Base, DeepSeek-V4-flash-Base and DeepSeek-V4-Pro-Base. Hy4 flash Base posts an 88.7 on MMLU. I don’t have the rival scores from the material, so I won’t pretend to rank anyone. But picking DeepSeek’s two V4 base models as yardsticks tells you who Tencent thinks it’s measured against.

And the architecture story hangs together. The repo credits LatentMoE, hybrid attention and Engram n-gram embeddings with shrinking KV cache size and compute while keeping benchmark results competitive. Smaller KV cache matters most at long context, and this model advertises 1M tokens. Why else build it that way?

I’d read the 1/13 and 1/20 figures as claims to test, not results to quote. A vendor-authored efficiency number is a starting bid. Still, an open repo with this much architectural detail is a bid that can be checked, which beats most launch-day marketing.

Pressure lands on anyone selling large models at higher per-task prices. If an open 12B-active model gets near the quality of pricier siblings, the argument for paying for bigger active footprints gets harder to make. That’s a conditional, not a verdict. Independent runs will decide it.

Where Hy4 flash sits in Hunyuan’s push toward sparse models

Tencent’s Hunyuan series has gone through several generations of open and proprietary models. Over that run the focus has drifted toward sparse architectures and efficiency work meant to cut inference costs. Hy4 flash looks like the sharpest expression of that direction so far, at least on paper.

It also sits inside a family. The repo compares it against Hy4 and Hy3 preview, so this isn’t a standalone model but one tier of a lineup. The “most cost-effective” label is a claim about position within the Hy range, not a claim about the whole market.

One practical gap: the repository page didn’t give us a release date we could verify, so I’m not putting one on this. What the page does give is architecture, parameter breakdowns and a benchmark table. That’s enough to describe the design. It isn’t enough to judge real-world quality, since the benchmark comparisons cover base models rather than deployed products.

It’s also worth separating two ideas people mash together. Total parameters set how much the model can store, and the Engram portion alone is about 64B of that. Active parameters set how much work happens per token. A model can be big on the first and light on the second, which is exactly the trade Tencent is advertising.

Three checks that will settle the efficiency story

First, watch for independent reproduction of the WorkIndex result and the cost-per-task math. If outside teams land near 1/13 and 1/20, the claim stands. If the gap narrows once the cost basis changes, that’s the story.

Second, look for head-to-head results against the DeepSeek-V4 base models the repo already names. The benchmark table is Tencent’s framing. Third-party numbers on the same models would show whether the efficiency comes with a quality price.

Third, track real deployment behavior: the 150 tokens-per-second figure, long-context performance near the 1M limit, and how the 220B total footprint behaves in practice. Cheap per token is one thing. Easy to host is another.

Editor's Note

What caught my eye wasn't the 220B headline, it was how carefully Tencent itemised the parameters. Most launches wave a total and move on. Here you can add up the backbone, Engram, vision and drafter yourself. I like that. But I'm holding the 1/13 and 1/20 cost claims at arm's length, because they're measured against Tencent's own models. I'll be watching for someone outside the company to rerun them.

– Sanket Chaukiyal, founder, SmartChunks

FAQ

What is Hy4 flash?

It's a multimodal mixture-of-experts model from Tencent-Hunyuan that Tencent calls its most cost-effective Hy model to date. It has 220B total parameters, takes text and image inputs, and supports up to 1M tokens of context.

How many parameters does Hy4 flash activate per token?

About 12B: 11.3B for text and 0.43B for vision. Each MoE layer holds 512 routed experts and activates 16 per token.

What does Tencent compare Hy4 flash against?

The base-model benchmark table compares Hy4 flash Base with Hy4 Base, Hy3 preview Base, DeepSeek-V4-flash-Base and DeepSeek-V4-Pro-Base. Hy4 flash Base scores 88.7 on MMLU.

Are the cost claims independently verified?

No. The WorkIndex score of 42.1 at 1/13 the cost per task of Hy4 preview and 1/20 of Hy4 comes from Tencent's repository. Treat it as self-reported until outside groups reproduce it.

Sanket Chaukiyal — Editor at Smart Chunks

Sanket Chaukiyal

Technology editor • 12+ years in editorial

Sanket is the founder and editor of Smart Chunks. He spent over six years at Autocar India (Haymarket SAC Publishing) as Sub Editor and Senior Copy Editor, and later served as Account Director (Content) at Rite Knowledge Labs. He holds a Master's in Media and Communication from the Symbiosis Institute of Media and Communication.

All articles → • LinkedIn