Skip to main content
Search

# Processing-in-Memory

1 related articles

Subscribe to updates
  1. arXiv

    CHARMING splits attention heads between tensor array and DRAM compute, lifting prefill 23.6% and decode 44%

    Transformer inference on edge devices is limited less by arithmetic than by moving weights and intermediate results between memory and compute. CHARMING splits attention heads across an AI Engine array and an in-DRAM compute unit that run concurrently, choosing which heads to offload under accuracy and quantization constraints; on a validation platform built from a VCK5000 and an Alveo U200, it improves prefill performance by up to 23.6% and decode by up to 44% over AIE-only CHARM, cuts per-layer energy by up to 7.6% and improves energy-delay product by up to 24.7% with fixed-point in-memory computation, while keeping accuracy similar to a 32-bit floating-point baseline.

You’re all caught up