Samsung has unveiled a new Compute Express Link (CXL) Add-in Card called the CXL Memory Module-Hybrid for Tiered Memory (CMM-H TM), which adds additional RAM and flash memory that can be remotely accessed by CPUs and accelerators. The expansion card comes with a mixture of high-speed DRAM and NAND flash and is intended to provide a cost-effective way to boost memory capacity for servers without resorting to locally-installed DDR5 memory.
Samsung's solution runs on the Compute Express Link (CXL) , an open industry standard that provides a cache-coherent interconnect between CPUs and accelerators, thus allowing CPUs to use the same memory regions as connected devices utilizing CXL. The remote memory, or in this case, a hybrid RAM/flash memory device, is accessible over the PCIe bus, which comes at the cost of ~170-250ns of latency, or roughly the cost of a NUMA hop. CXL was introduced in 2019 and is in its third revision right now, featuring PCIe 6.0 support.
CMM-H TM is an offshoot of Samsung's CMM-H CXL memory solution. Samsung says it is the world's first FPGA -based tiered CXL memory solution and is designed to "tackle memory management challenges, reduce downtime, optimize scheduling for tiered memory, and maximize performance, all while significantly reducing the total cost of ownership."
This new CMM-H isn't as fast as DRAM; however, it adds a beefy slab of capacity via the flash but hides a lot of latency with a clever memory caching feature built into the expansion card. Highly used data is moved to the card's DRAM chips to improve speed, while less used data is stored in NAND storage. Samsung says this behavior happens automatically, but some applications and workloads can give the device hints to improve performance through an API.
Samsung's new expansion card will provide its customers with new ways to expand their sever's memory capacity outside of DDR5. This new design paradigm is becoming more important as more advanced large language models continue to demand more and more memory out of their host machines and accelerators.
