Zyphra Introduces Hierarchical Attention ZHA: A Path To True Subquadratic Attention
Zyphra introduces Zyphra Hierarchical Attention, a sparse attention mechanism designed to avoid the quadratic computation costs of transformer models. Surprisingly,...