AMD Strix Halo 128GB Unified Memory Loads 70B Models at 256 GB/s Where RTX 5090 32GB GDDR7 Cannot

Unified memory architectures merge CPU, iGPU and NPU address spaces into a single soldered LPDDR5X pool, removing the PCIe transfer step that confines discrete GPUs to their onboard VRAM. AMD, Apple, Intel and Qualcomm SoCs now ship this layout at 128GB capacity for under $2000, a threshold discrete cards reach only at $8500 workstation pricing. The shift directly enables local 70B inference on edge hardware that previously required cloud clusters.

Vendor datasheets and owner traces show bandwidth remains the limiter. Strix Halo peaks at 256 GB/s, M4 Max at 410-546 GB/s and RTX 5090 at 1792 GB/s; real delivered figures sit 15-25 percent lower. Roofline analysis from Williams, Waterman and Patterson demonstrates token generation becomes memory-bound below roughly 400 GB/s for 70B workloads, producing the observed slow-reader speeds on mini PCs despite full model residency.

Operationally this trades throughput for accessibility. Organizations gain on-premise 70B capability without datacenter power or procurement cycles, expanding edge deployment in regulated sectors. GPU cluster dominance erodes for inference once model size exceeds single-card VRAM, favoring dense unified-memory nodes for privacy-sensitive or disconnected workloads.

Next hardware revisions target bandwidth uplift via wider buses and higher LPDDR5X clocks while retaining capacity scaling; Apple M3 Ultra already demonstrates 819 GB/s at 512GB, indicating the trajectory for 2026 mini-PC SKUs.