Baidu researchers have built an OCR model that handles dozens of document pages in a single inference pass, keeping memory use and speed constant regardless of text length. A redesigned attention mechanism makes this possible.
No current OCR model handles more than about ten pages in a single pass, the Baidu researchers write in their technical report. The bottleneck is the KV cache, a buffer where a language model stores all previously processed tokens during generation so it can look them up later.
Current end-to-end systems use a language model as their decoder, so this buffer grows with every new line of text. That drives up memory use and steadily slows generation. In practice, systems get around the problem with a loop that processes each document page by page, resetting the cache after every step.
Baidu frames the problem with a human analogy. Someone copying a book doesn’t re-read everything they’ve already written. They keep their eyes on the source, the last few characters they wrote, and the next one to put down. Older passages fade through a kind of soft forgetting. The researchers want Unlimited OCR to mimic that pattern.
A fixed window caps memory use
It works through what the team calls Reference Sliding Window Attention (R-SWA). Each generated token still sees all reference tokens, the visual image tokens and the prompt. But when it comes to previously generated output, it only looks back at the last 128 tokens. That keeps the KV cache constant throughout the entire process instead of growing linearly with output length.
Standard sliding window attention would also subject visual tokens to ongoing state changes, gradually blurring image features and degrading recognition. R-SWA exempts visual tokens from these transitions. They’re encoded once and stay unchanged.
The KV cache works as a queue where each new token pushes out the oldest one. With standard multi-head attention, memory use grows without bound as token count rises. R-SWA caps it at the fixed sum of prefix length and window size.
Built on top of Deepseek OCR
Unlimited OCR builds on the open-source Deepseek OCR model. Baidu keeps its DeepEncoder and pairs it with a mixture-of-experts architecture with three billion parameters, of which only about 500 million are active during inference. The DeepEncoder compresses a 1024-by-1024-pixel PDF image down to 256 tokens.
Two resolution modes carry over. “Base” mode handles multi-page documents, and “Gundam” mode uses dynamic resolution for single pages. Every standard attention layer in the decoder was swapped out for R-SWA.
Training used about two million document samples, split 9-to-1 between single-page and multi-page data. All data was packed into sequences of 32,000 tokens; training ran for 4,000 steps on 8 times 16 Nvidia A800 GPUs. The DeepEncoder stayed frozen, and only the language model parameters were updated.
Better scores despite limited attention
Unlimited OCR scores 93 percent overall on the OmniDocBench v1.5 document benchmark, six percentage points above the Deepseek OCR baseline, according to the authors. On the newer v1.6 version, the model hits 93.92 percent, putting it at the top of the end-to-end system rankings.
In the long-horizon test, where the model processes many pages in a single pass, the error rate stays below 0.11 even past 40 pages.
The constant cache also pays off in speed. In Base mode, Unlimited OCR hits 5,580 tokens per second versus 4,951 for Deepseek OCR, a 12.7 percent bump. In a theoretical comparison of upper bounds with ideal parallelism, the model leads the baseline by 35 percent at around 6,000 output tokens.
Not truly unlimited yet
The model’s fixed context length of 32,000 tokens limits how many pages it can take in, since visual tokens stack up with each additional page. Baidu plans to train 128,000-token models soon and eventually build a prefill pool that lets the model fetch relevant KV blocks on its own, like flipping through a book. The authors also see R-SWA as transferable to other reference-based tasks like speech recognition and translation.
Code and model weights are on GitHub and Hugging Face. The model runs on ModelScope and the inference engines vLLM and SGLang.