Loading...
Loading...
Loading...
Tech ExplainedSeptember 6, 20237.1 MIN READ

Leveraging Compute Express Link (CXL) at the Edge

Compute express link

What is Compute Express Link?

Compute Express Link™, or CXL™, is an open standard for high-speed CPU connection to memory and other devices. At its core, CXL has its roots in addressing the long-sought desire to achieve a modular and dynamically composable and scalable computational system. For example: 

  • Need more RAM? Drop in some memory expansion modules. 

  • Need processing cores? Add another processor module. 

  • More connectivity? Incorporate a new network module. 

…and oh by the way, it’s all hot-pluggable, so no need to bring down the system for maintenance, or reboot to have the new hardware recognized.Here’s how the CXL consortium website puts it:

“Compute Express Link™ (CXL™) is an industry-supported Cache-Coherent Interconnect for Processors, Memory Expansion and Accelerators. CXL technology maintains memory coherency between the CPU memory space and memory on attached devices, which allows resource sharing for higher performance, reduced software stack complexity, and lower overall system cost. This permits users to simply focus on target workloads as opposed to the redundant memory management hardware in their accelerators. 

CXL is designed to be an industry open standard interface for high-speed communications, as accelerators are increasingly used to complement CPUs in support of emerging applications such as Artificial Intelligence and Machine Learning.”

What is keeping CXL from widespread use?

Conceptually, this all sounds great, but it is important to note that this concept has been slowly developing and has been on the minds of many computer engineers over the past several decades. Strides have been made in the direction of this dream, inching ever closer to the goal, but one of the biggest barriers to date has been the effective throughput and bandwidth capabilities of various bus architectures and interconnects. 

This bandwidth limitation impacts the ability of shared memory spaces (such as cache) to be maintained coherently (e.g. updated in multiple places at the same time), which is critically important when extremely large workloads requiring significant memory consumption have to be processed across multiple CPUs (or multiple accelerators such as GPUs, DPUs, VPUs).

Unfortunately, without a fast enough bus, it is difficult to maintain the necessary memory coherency amongst a primary host processor’s memory and devices. 

That said, this is where proprietary vendor specific cache coherent buses like Intel’s Ultra Path Interconnect (UPI), AMD’s eXternal Global Memory Interconnect (XGMI), and Nvidia’s NVLink have addressed some of the coherency and scalability issues.

With the bandwidth potential and speed of capabilities of PCIe 5.0 at 32GT/s per lane, and soon PCIe 6.0 at 64GT/s per lane, the bandwidth barrier is breaking down. That improvement in capabilities, along with the vendor specific methodologies of enabling data coherency and scalability, are paving the way for the next generation of an open standards-based scalable computation method via Compute Express Link.

The CXL standard is implemented and documented by the CXL consortium, and the consortium  releases the features, functions, and capabilities of CXL as a revision controlled specification (not unlike what the USB Integrators Forum does for the USB standard).

Compute Express Link Versions & Capabilities

To date, the CXL consortium has released the following specifications:

  • CXL 1.0

  • CXL 1.1

  • CXL 2.0

  • CXL 3.0

Compute Express Link 1.0

CXL 1.0 is the initial CXL specification that was released in March of 2019. This specification defines a standard way of connecting to I/O, caching, and memory subsystems agreed upon and supported by 170 companies. CXL leverages the PCIe Express® (PCIe®) physical bus at Gen 5.0 (and down to Gen 3.0 in a degraded mode) and is comprised of three protocols:

  • CXL.IO – For Initialization, Link up, Device Discovery/Enumeration, Register Access

  • CXL.MEM – Allows CXL devices to cache host memory with low latency

  • CXL.CACHE – Allows host processor to access device attached memory

With these three protocols, CXL can be enabled for 3 device types as follows:

A breakdown of the types of devices CXL can be enabled for and the protocols used.

Compute Express Link 1.1

CXL 1.1 was released in September 2019 with some errata and additional context pertaining to CXL compliance. 

CXL 1.1 is the first specification that is being widely implemented. CXL 1.1 capabilities are available on Intel 4th Generation Xeon (formerly known as Sapphire Rapids) and 4th Generation AMD EPYC™ Processors (formerly known as Genoa).

Compute Express Link 2.0

CXL 2.0 was released in November 2020. Fundamentally, the major additional feature enabled by the CXL 2.0 specification is the support for connectivity over a single level switch, allowing for fine-grained dynamic (hot plug) resource assignments of CXL devices to the available host(s) via a fabric manager, all while maintaining full backwards compatibility with CXL 1.1. Additionally, telemetry, load management, and additional security features were also added with CXL 2.0.

Compute Express Link 3.0

CXL 3.0 was released in August 2022 and is targeted to continue the path of the CXL 2.0 specification by further enabling the scaling functionality of CXL. 

CXL 3.0 is based on PCIe 6.0 technology which doubles the transfer rate of PCIe 5.0 to 64GT/s with no additional latency and enables up to 256GB/s data transfer on a x16 link.

Key features of CXL 3.0 include:

  • Enhanced Coherency – The ability to back invalidate the Host’s caches for Type 2 devices (accelerators with memory).

  • Memory Pooling and Sharing – The ability to treat CXL-attached memory as a fungible resource that can be flexibly allocated and deallocated to different systems depending on the need.

  • Fabrics – The ability to go beyond traditional tree-based architectural structures of PCIe and previous CXL generations, which allows for Global Fabric Attached Memory (GFAM) devices (e.g. A GFAM device is like a traditional CXL Type-3 device, except it can be accessed by multiple nodes).

CXL Feature Summary by Specification Release

Features

CXL 1.0/1.1

CXL 2.0

CXL 3.0

Release date

2019

2020

2022

Max link rate

32 GT/s

32 GT/s

64 GT/s

Type 1, 2 and 3 devices

X

X

X

Memory Pooling w/MLDs

X

X

Global Persistent Flush

X

X

CXL IDE (Integrity and Data Encryption)

X

X

Switching (Single-level)

X

X

Switching (Multi-level)

X

Direct memory access for peer-to-peer

X

Enhanced coherency (256 byte flit)

X

Memory sharing (256 byte flit)

X

Multiple Type 1/Type 2 devices per root port

X

Fabric capabilities (256 byte flit)

X

How is CXL being used today?

As CXL 1.1 has just become available from a datacenter class CPU perspective starting in late 2022, CXL is in the very early stages of being implemented in major data centers (such as those that are owned and operated by major cloud service providers like Microsoft and Amazon).

Additionally, it is worth noting that while major CPU suppliers are now supporting CXL 1.1, for CXL to operate, it must also be supported by the endpoint (e.g. device) suppliers.

Therefore, based on the newness of CXL functionality, we find ourselves in an exciting time where implementations are just now starting to become reality.

Many speculate that in the data center, the impact CXL will have on memory utilization is going to be profound. In fact, many major industry leaders perceive that a significant portion of system memory (upwards of 25%) is stranded (unused) at any given point in time.

How could CXL be used at the edge?

It is fair and reasonable to expect that CXL usage at the edge will lag that of the data center, specifically noting that CXL was designed with the data center in mind. However, that is not to say that the features and capabilities that are being enabled by CXL will not eventually find value at the edge for scenarios where high performance, on-premise compute is required.

For example, the cache coherency afforded by CXL for Smart-NICs (Type 1 Devices), GPUs, and other accelerators (Type 2 Devices) could be profoundly beneficial to edge computing performance.

Additionally, memory expansion (Type 3 Devices) could bring increased flexibility and longevity for compute devices at the edge such that existing devices could be expanded and enhanced over time, not merely replaced.

For example, as artificial intelligence (AI) continues to become more ubiquitous, inference models sizes are growing at a rapid pace. The larger the inference model, the more memory required. Generative AI models that have billions of parameters have already begun to exceed what can be supported by the amount of memory found on some of the most powerful GPUs on the market. In the future, as we add new AI capabilities at the edge, adding additional memory available to a GPU or CPU without replacing the processor could become incredibly valuable.

The possibilities are seemingly endless and overwhelming, but at OnLogic, we’re excited for the future and all that new technologies, like Compute Express Link, will bring to the table.

This article was first published on LinkedIn on July 11, 2023.

Author Image of Ross Hamilton
Ross Hamilton
Senior Solution Architect , OnLogic
Ross is a Senior Solution Architect at OnLogic responsible for research, strategy, and product innovation. When he's not at work, Ross likes to spend time with his family enjoying movies/television/games, and care for his many animals.

Stay in the know.

Get first access to the newest products and Industry 4.0 innovation, OnLogic Live! events and more.