Baidu’s Unlimited OCR replaces decoder attention with Reference Sliding Window Attention (R-SWA). Each token attends to all reference tokens (visual + prompt) while limiting output attention to the preceding 128 tokens, keeping KV cache constant during decoding.
Built on DeepSeek OCR baseline via continue-training. 3B-parameter MoE with 500M active parameters. Transcribes dozens of pages in one 32K forward pass. Scores 93.23 on OmniDocBench v1.5 (+6.22 over baseline) and 93.92% on v1.6. Throughput reaches 5,580 tokens/second in Base mode. Code and weights at github.com/baidu/Unlimited-OCR. R-SWA applicable beyond OCR to ASR and translation.