AI inference is increasingly bottlenecked by memory, not just compute. As workloads move toward longer conversations and agentic reasoning, the key-value (KV) cache — which stores intermediate results for reuse — expands rapidly. GPU memory and system memory are fast but costly and limited, while traditional storage offers capacity but adds CPU-mediated latency that can leave accelerators idle.
According to a sponsored post on The Register, Huawei's answer is the OceanStor M900 Context Memory Storage platform, introduced at HUAWEI CONNECT 2026. It extends the KV cache beyond accelerator and system memory into a shared, multi-tier storage pool connected by Huawei's UnifiedBus interconnect. The company claims up to 64PB of capacity, 60-microsecond access latency, and single-hop NPU access via its ASU disk, which integrates CPU, networking, and storage controller functions.
Huawei also highlights durability and cost: KV-aware adaptive storage manages data placement by cache lifecycle, with claimed endurance of up to 24 drive writes per day — 16 times that of conventional SSDs. The broader pitch is that storage, memory, networking, and compute must be treated as one interconnected system for inference performance. Since this is a sponsored post, these figures should be read as Huawei's own assertions rather than independent benchmarks.