Parallax: A Parameterized Local Linear Attention That Keeps Softmax and Adds a Learned Covariance Correction Branch

Parallax replaces LLA's per-query solver with a learned projector, doubling arithmetic intensity and improving perplexity at 0. 6B and 1. 7B. The post Parallax: A Parameterized Local Linear Attention That Keeps Softmax and Adds a Learned Covariance Correction Branch appeared first on MarkTechPost.