As language models (LMs) improve at tasks like image generation, trivia questions, and simple math, you might think that ...
Nexus proposes higher-order attention, refining queries and keys through nested loops to capture complex relationships.