Skip to main content
Search

# Bilinear Rank

1 related articles

Subscribe to updates
  1. arXiv

    Swapping matrix multiplication for an associative-algebra product: a 110M-parameter model gains 6.2–7.8% generation throughput while GSM8K, MBPP and IFEval all drop

    The work replaces ordinary matrix multiplication in Transformer projections with an associative-algebra multiplication table that keeps the full learned weight bank and parameter count, lowers the bilinear rank for the q=2 case from 7 (Strassen's 2×2 algorithm) to 6, and in a controlled pretraining run of two approximately 110M-parameter models over 12.3B tokens observes a 6.2–7.8% end-to-end generation throughput gain across four prompt domains together with lower scores on GSM8K, IFEval and MBPP than the dense baseline.

You’re all caught up