DeepSeek-V4.1 Flash: Pushing the Limits of KV Cache Compression

TL;DR

When DeepSeek-V4.1 Flash was released, I thought it might just be a post-training iteration version… but after using it for a while, I found it reached nearly 420 Tokens/s in speed, and then Cui said all DeepSeek-V4 Pro models would be taken offline… suddenly I felt this was no small matter… until the Technical Report “DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression” was fully released, only then did I realize it should be called DeepSeek-V5 Flash…

As the paper title states, the purpose of DeepSeek-V4.1 Flash is to push KVCache compression to the extreme. The main reason is that Long-horizon Agent Workflows cause the Context to grow longer and longer, while various tool calls bring heavy prefill computation pressure. The storage pressure of KVCache in HBM and external SSD is very high, all of which are reasons that make Scaling impossible.

Key architectural optimizations:

• Prefill computation optimization: Drawing on YOCO, the entire model has 40 layers, and only 20 layers are needed during Prefill. Therefore, the Prefill activated parameters are only 8B, and the Decode activated parameters are 16B.
• KVCache compression: Engineering-wise, KVCache compression is divided into several dimensions: head count compression similar to GQA, then block-based compression like CSA, and the cross-layer compression of CSA2 in this paper. At the same time, the indexer computation of Sparse Attention is also optimized. Finally, there are some numerical precision optimizations, for example DS41F adopts FP4 KVCache.

Finally, under the premise of maintaining high-quality task completion by the model, KVCache is further compressed by 4x.

Why KVCache compression is needed

In recent years Long-horizon Agents have made ultra-long-context processing an increasingly important model workload. Supporting this type of workload not only requires efficient processing of long sequences, but also requires persistent storage, reuse, and transfer of large KVCache.

DeepSeek-V4.1-Flash is a multimodal mixture-of-experts model designed for more aggressive KVCache compression. It has a parameter scale of 552B, natively supports multimodal input, and supports contexts of up to 1 million tokens. It adopts a Causal Encoder-Decoder (CED) architecture, in which the Decoder’s Global KVCache is obtained by projecting the Encoder’s final hidden states. This design makes the model activate 8B parameters per token during the Prefill stage and 16B parameters during the Decoding stage.

Although DeepSeek-V4.1-Flash is significantly larger than DeepSeek-V4-Flash, at the same sequence length, its required runtime KVCache storage is only about 1/4 of the latter, and its persistent KVCache storage is only about 1/8 of the latter. In addition, the overall performance of DeepSeek-V4.1-Flash is superior to DeepSeek-V4-Flash.

CSA2 compression dimensions

• In the channel dimension, a 512-dimensional latent vector is used to share the representation of the keys and values required by each attention head.
• In the sequence dimension, the Encoder merges 2 adjacent positions into 1 cache entry through channel-wise learned weights, while the Decoder retains per-position entries.
• In the layer dimension, multiple layers share the same global KV, and the whole network retains only 3 copies of Encoder cache and 1 copy of Decoder cache.

Combined with FP4 quantization, the storage growth of the global main KV and the Indexer is about 890 bytes per token.

Model architecture overview

The backbone parameters are about 552B, with Engram parameters additional. Activated parameters per token for prefill and decode are about 8B and 16B respectively. The model has 40 layers total, hidden dimension 5120, vocabulary size 129280. Each layer contains attention and MoE, organized through mHC residual connections.

Key components:

• CED (Causal Encoder-Decoder): First 20 layers are Encoder, last 20 are Decoder. Decoder’s global KV comes from Encoder end representation.
• CSA2: Local sliding window (128) + global sparse retrieval + cross-layer KVCache reuse.
• DSpark: Speculative decoding with SWA-128 draft blocks.
• Engram: Conditional memory injected at layers 1 and 14.
• Vision branch: 32-layer ViT with patch size 14, 3×3 downsampling for 9x token reduction.

MoE: 384 routed experts, top-6 activated, moe_inter_dim 2304.