PENSA runs BitNet b1.58 2B4T end to end on an Alveo U50: 58.09 tok/s at context 128, 2.14–3.14 times a same-model Xeon baseline
Synopsis
Ternary large language models push linear-layer weights to three values, replacing multiplication with sign-controlled addition and shifting the bottleneck to packed-weight delivery and reduction. PENSA executes the unmodified BitNet b1.58 2B4T end to end on an AMD Alveo U50 FPGA, with two ternary matrix engines reading TQ1_0 packed weights directly from high-bandwidth memory and each processing 1,280 ternary weights per cycle. At 104.1 MHz, two-stream interleaving reaches 58.09 tok/s at context 128 and 24.60 tok/s at context 2048, 2.14–3.14 times a same-model Xeon E-2356G baseline.
Interpretation
PENSA keeps the entire autoregressive inference of the 2B-scale BitNet b1.58 2B4T on a single AMD Alveo U50 FPGA without changing the model structure. Software ternary inference such as Bitnet.cpp relies on lookup-table and multiply-add CPU kernels, while FPGA ternary accelerators have mostly used lookup-based matrix multiplication, sparse dataflows or compute-memory co-design; end-to-end execution of BitNet b1.58 2B4T with direct packed-weight execution has been comparatively unexplored. At the achieved post-route frequency of 104.1 MHz, two-stream interleaving measures 58.09 and 24.60 tok/s at contexts 128 and 2048, corresponding to 2.14–3.14 times the same-model Xeon E-2356G baseline.
Weights stay in the model's native TQ1_0 ternary format in high-bandwidth memory, packing five ternary values per byte for a 1.6-bit-per-weight payload. The two ternary matrix engines split output rows evenly, each drawing weight stripes from four independent high-bandwidth memory pseudo-channels, and the sequencer merges their complementary output rows. Each 512-bit AXI beat carries two 256-bit TQ1_0 blocks of 160 ternary weights, so four interfaces deliver 1,280 ternary weights per cycle to each engine; a compile-time 256-entry decode ROM maps each byte to five ternary controls that select add, subtract or zero.
Two-stream interleaving advances two independent sequence slots alternately through the persistent pipeline, so one slot can move through transformer layers while the other occupies the vocabulary projection. Single-stream execution leaves idle time in the persistent units, and interleaved scheduling fills it. Across the evaluated contexts, interleaving raises aggregate throughput to 1.37–1.71 times single-stream, but it does not amortize the vocabulary-projection weight stream across sequences.
Perspective
The demonstrated scope is the unmodified BitNet b1.58 2B4T on a single AMD Alveo U50, covering decode throughput from context 128 to 2048, prefill time, power and model fidelity; cross-accelerator figures serve as architectural positioning, not direct performance comparison.
Decode latency depends strongly on context: at context 2048 the context-dependent term contributes about 60% of modeled latency, with attention traversal accounting for 97.3% of its fitted per-key component; prefill grows from 6.55 s at 128 tokens to 109.18 s at 2048 tokens; throughput is locally non-monotonic between contexts 256 and 512, and the present measurements do not isolate this deviation.
