Skip to main content
Search

# GPU memory oversubscription

1 related articles

Subscribe to updates
  1. arXiv

    When GPU memory runs out, stop paging: Ember keeps cold data on the host and speeds LLM generation several-fold

    NVIDIA's Unified Virtual Memory collapses once data exceeds GPU memory, and Ember inverts the default: allocations start mapped into host memory, GPU hardware access counters decide which regions turn hot enough to promote, and cold regions are reclaimed in batches before memory fills. Across six applications on three PCIe GPUs under oversubscription, Ember reaches geometric-mean speedups over stock UVM and sustains LLM generation at several times UVM's throughput, with no profiling run or application changes.

You’re all caught up