Tencent Hunyuan open-sources Hy4 preview: 770B total parameters, 49B active per token, edging GLM 5.3 and Kimi K3 in a blind engineering-task review
Synopsis
Tencent's Hunyuan team has open-sourced Hy4 preview, a new mixture-of-experts model, together with an FP8 quantized version; it carries 770B total parameters with 49B activated per token and a 1M context length. The team calls it the largest generation-over-generation gain it has measured and places it at the open-source frontier. In an internal blind evaluation, 163 experts scored outputs on 203 engineering tasks, and Hy4 preview averaged 2.99, slightly above GLM 5.3 at 2.92 and Kimi K3 at 2.94.
Interpretation
The Tencent Hunyuan team released the Hy4 preview weights along with an FP8 quantized version on Hugging Face, ModelScope, GitCode and CNB under the Apache 2.0 license. The team describes this generation as the largest capability jump it has measured and says it is enough to put Hy4 preview at the open-source frontier; that judgment is the team's own framing rather than a published comparison against external baselines. The supporting comparison is a team-run blind evaluation in which 163 internal experts scored model outputs side by side on 203 engineering tasks: Hy4 preview averaged 2.99, GLM 5.3 2.92 and Kimi K3 2.94.
The model uses a mixture-of-experts backbone in which the first of 78 layers keeps a standard dense feed-forward network and the remaining 77 layers each hold 256 routed experts plus one shared expert, with every token activating the top-8 routed experts and the shared expert. The attention module uses Gated DeepSeek Sparse Attention with IndexCache for cross-layer sparse index reuse, and the residual pathway uses iHC to widen inter-layer information flow; the team says both choices were inspired by DeepSeek and GLM. The published specifications list a hidden size of 6144, 64 attention heads, query and key-value compression dimensions of 2048 and 512, a 1M context length and a 120832-token vocabulary, plus one native MTP layer for speculative decoding with 10B total and 0.7B activated parameters.
Perspective
The demonstrated scope is the team's own blind evaluation over 203 engineering tasks, the published specification table, and the weights and FP8 version on four platforms; the team also states this is an early Hy4 version with headroom left in both pre-training and post-training.
The team's listed known issues include spending longer than necessary reasoning through complex tasks and a tendency to over-verify its own work; it says it will iterate quickly and continue the Hy3 preview practice of shipping early to hear what breaks.
