Skip to main content
Search

Kazi Ashik Islam

1 related articles

Subscribe to updates
  1. arXiv

    Bilateral log-linear attention adds an origin-anchored hierarchy: MQAR mean rises from 76.37% to 96.12%, and the 6144-token retrieval mean nearly doubles

    Log-linear attention compresses older context more coarsely and keeps fine-grained summaries only near the query. Bilateral log-linear attention adds a second hierarchy anchored at the sequence origin alongside the existing Fenwick hierarchy, so both early and recent context get finer reads. On MQAR with matched depth and hidden width, mean overall test accuracy rises from 76.37% to 96.12% across ten training seeds; under approximate parameter matching, the six-task RULER-NIAH mean at 6144 tokens rises from 26.97% to 50.03%.

You’re all caught up