AI Server Architecture: CXL, DDR5, PCIe 6.0 and Ethernet
In response to the development trends of AI/Data center/HPC and other technology applications, server-related technologies are keeping pace to meet their high-speed computing and transmission requirements. The development goals are all directed at low latency, high reliability, and improved power efficiency. For example, CXL, which has been gaining attention recently, is used to overcome the limitations and low utilization of bandwidth available to server processors (CPUs) for interconnecting memory. The CXL protocol is designed for high-speed signal transmission applications to establish cross-chip memory interconnection and shared access modules to solve the current performance bottleneck between processors and memory in servers.Credit: Boardcom
1.CXL:
It facilitates seamless high-speed communication between processors (such as CPUs, GPUs, and FPGAs) and accelerators, enabling them to work together efficiently and effectively. CXL3.0/3.1 is capable to provide higher bandwidth and transmission speed that meets massive data transfer of the rapid pace of technology development. CXL 3.0/3.1 features memory sharing that allows individual memory regions within pooled resources to be shared between multiple hosts.. This innovative approach reduces data movement, leading to low latency and better performance . Additionally, memory consistency ensures that all accelerators and processors have a coherent view of the memory pool, further enhancing system efficiency.
there are two main types of products in the development of CXL: "Memory Expansion Modules" and "Memory Pools." The actual product implemented on the market is the "Memory Expansion Module." The composition of a CXL memory expansion module includes a CXL controller chip along with DRAM memory chips. The DRAM memory inside the module is connected to the server CPU via the CXL controller chip through the PCIe interface. The CXL memory expansion module is designed for single-server environments. However, memory pooling enables the maximization of memory capacity utilization.
Through CXL switch interconnect technology, memory from multiple servers is combined to form a shared memory pool, overcoming traditional single-server limitations to meet future technological demands.
Credit: CXL Consortium
Credit: Samsung
2.DDR5:
The features of DDR5 have been further optimized than DDR4 to be more capable to process hyperscale of data, more complex computing of innovative technology, it includes enhanced operating efficiency, less power consumption, higher bandwidth, higher density and larger capacity of memory. It’s especially mentioned that DDR5 fully integrates its own Error-Correcting Code (ECC) functionality, allowing for more effective error detection and correction. This enhances the reliability and stability of systems using DDR5 memory.
For example, we provided the customer DDR5 Post-sim simulation service as following, by layout schematics, we simulated relevant parameters such as return loss and insertion loss. and go with the corresponding signal I/O component models for simulation analysis
DDR5 Topology
After adjusting and re-simulating the original design, the eye diagram results improved, meeting design specifications. For those interested in learning more about simulation analysis, please refer to the videos: "simulation analysis 101"and"SI/PI Simulation Analysis."
3.PCIe 6.0 :
PCIe serves as the physical layer role, providing the I/O interface for connecting expansion cards and external hardware devices. Via CXL, accelerators like GPU/FPGA can connect to the system and share memory. The CXL 3.0 specification has been officially released, with the critical feature being the adoption of PCIe 6.0 based on PAM4 encoding at the physical layer, doubling the transmission bandwidth and speed.
4.Ethernet :
Ethernet: Not only CXL, but the network interface cards (NICs) used in servers are also provided via PCIe interface to establish Ethernet connections, enabling remote access and memory sharing through 100G/400G/800G/1.6T Ethernet.
With technologies like CXL, PCIe, and DDR continually are developing latest generation to meet the demands of recent data centers for high transmission speeds and efficiency. These efforts enable chipset, memory, and high-speed interfaces to develop symbiotic relationships during the whole operation, it’s as well as the significant progress for data transmission speed, memory utilization, storage efficiency, and overall system performance.
These technologies are critical elements for data centers/servers to achieve higher speeds, lower latency, and less power consumption. iPasslabs is able to provide professional one-stop test services for these technologies, including simulation in the early design stages, validation during the R&D phase, and test fixtures. With the experienced iPasslabs team involved in the product development process, they can also provide professional advice, maximizing efficiency and effectiveness.
Reference: https://www.ithome.com.tw/tech/153366
Reference: https://www.broadcom.com/products/pcie-switches-retimers/expressfabric
entrance picture credit：Google