Definition
Attention-free sequence architecture (Zhang Chong, arXiv 2512.19428): reduce token states to rank r, encode local pairs as 2D subspaces on a Grassmann manifold via Plücker coordinates, fuse back with gated mixing—O(L) sequence cost for fixed r vs quadratic attention.
Key Points
- Paper: ~10–15% Wikitext-2 perplexity gap vs size-matched Transformer (13–18M params); slight SNLI head gain on DistilBERT (2026-10-26-grassmann-flows-arxiv-html)
- Independent reproduction reports ~22.6% test perplexity gap—wider than claimed (2026-10-26-grassmann-flows-reproduction-github)
Preprint, small benchmarks only—not evidence that attention is obsolete.