Zyphra Hierarchical Attention, or ZHA, is a research method for finding relevant information in very long AI contexts. Those figures at millions and billions of tokens are projections from a performance model, not measured end-to-end application speeds.
Searching a tree instead of a flat list
In its November 2 explanation, Zyphra describes arranging compressed summaries of prior information in a tree. A new query expands promising branches and selects a smaller set of entries for attention. It is a little like using a book’s contents and sections to locate a passage before reading it closely.
The method can open several branches when a question needs information from distant parts of the context. The full cache remains at the leaves of the tree, so the approach is not limited to a fixed-size selection.
Experiments and projections are different results
The researchers adapted an existing pretrained model by freezing its weights and training a small indexing component. They compared results on retrieval, reasoning and coding evaluations. These evaluations do not establish that the extreme-context speedups have already been measured in a deployed product.
Zyphra’s GPU performance model projects about 32–35 times faster attention than dense attention at ten million tokens and roughly 373–417 times at one billion. Those estimates concern the attention calculation under the model’s assumptions, not a measured speedup for an entire application.
