Berkeley Out-of-Order Machine (BOOM) v4 设计说明

总体概述

BOOM(Berkeley Out-of-Order Machine)是加州大学伯克利分校开发的开源高性能乱序执行RISC-V处理器内核,支持RV64GC指令集docs.boom-core.org。BOOM采用了**统一物理寄存器文件(PRF)**的设计,即通过显式重命名将架构寄存器映射到比架构寄存器数更多的物理寄存器上,实现对写后写(WAW)和写后读(WAR)相关的消除docs.boom-core.orgdocs.boom-core.org。这种设计与MIPS R10000和Alpha 21264等经典乱序处理器相似docs.boom-core.org。BOOM通过Chisel硬件构造语言编写,具有高度参数化特性,可以视作一系列同族微架构而非单一配置。

在微架构上,BOOM的流水线概念上可分为10个阶段:取指解码寄存器重命名派遣发射寄存器读取执行存储访问写回提交chipyard.readthedocs.io。不过实现中为了优化性能,这些阶段有合并:BOOM实际实现为7级左右的流水,如“解码/重命名”合并、“发射/寄存器读取”合并等chipyard.readthedocs.io。整个处理器可分为前端(取指及分支预测)和后端(乱序执行核心,包括重命名、调度、执行、提交)两大部分,它们通过取指缓存和队列衔接。

BOOM集成在Rocket Chip SoC框架中,复用Rocket的许多组件(如L1缓存、TLB、页表遍历单元等)docs.boom-core.org。下面将按模块详细说明BOOM v4的设计,包括各模块功能、关键数据结构和类、模块间交互以及完整的指令流水流程。

前端:取指与分支预测

BOOM的前端负责从指令缓存取出指令,并进行分支预测以保持流水线尽可能满载。BOOM使用了自研的前端模块(BoomFrontend),Rocket Chip的Rocket前端只是提供了I-cache等基础结构docs.boom-core.org取指过程如下:

  • 指令缓存(I-Cache): BOOM复用了Rocket Core的指令缓存实现。I-Cache是一个虚地址索引、物理地址标记的集合相联缓存docs.boom-core.org。每周期前端根据当前PC从I-Cache取出一个对齐的指令块,并将其暂存,以便后续解码使用docs.boom-core.org。I-Cache命中后提供指令位串;若未命中则触发访存请求,前端将停顿等待指令返回。
  • 取指宽度与取指包(Fetch Packet): BOOM支持超标量取指。每周期前端可取出一组指令,称为一个“取指包”,其大小等于前端取指宽度(例如2或4条指令)docs.boom-core.org。取指包中除了指令本身,还包含有效位掩码(指示该包中哪些字节是有效指令,例如应对RVC压缩指令)以及基本的分支预测信息docs.boom-core.org。这些信息将用于流水线后段的分支处理。
  • Fetch Buffer: 前端包含一个取指缓冲区,暂存取出的取指包docs.boom-core.org。取指包从I-Cache出来后进入该缓冲区,以解耦取指与解码阶段。如果解码或后端暂时阻塞,取指缓冲可以暂存多个取指包,避免前端I-Cache停滞。解码阶段将从取指缓冲区提取指令。
  • Fetch Target Queue (FTQ): BOOM前端还维护一个取指目标队列(FTQ),用于跟踪流水线中各取指包对应的PC、分支预测信息等元数据docs.boom-core.org。每当前端取走一个新的取指包,就将其起始PC、末尾预测的下一个PC或分支目标等信息记录到FTQ。FTQ的存在使得当后端检测到分支预测失误、异常等需要改变控制流时,能够快速找到对应取指包并提供恢复信息(例如正确的下一PC)。FTQ有效地充当了前端和后端之间关于控制流信息的接口。

分支预测对于维持高性能至关重要。BOOM前端在取指流水线中嵌入了多级分支预测器,尽量在取指当下周期就对可能的分支做出预测,从而“抢先”更新PC,减少取错指令的浪费。BOOM的分支预测主要包括:

  • 静态BTB&BIMODAL预测: BOOM包含一个**Branch Target Buffer (BTB)*用于缓存最近遇到的分支地址及其目标,提供直接的PC跳转。配合BTB的是一个简单的*双模(Bimodal)分支预测器,使用全局历史或局部历史记录去预测分支方向(Taken/Not taken)。双模预测提供了快速但相对粗粒度的方向预测。
  • TAGE/Tournament预测: 为提高复杂分支的预测精度,BOOM还实现了更先进的TAGE(Taggeed Geometric Predictor)和/或**组合赛选(Tournament)**预测器。在v4中,ifu/bpd/包下包含多个预测器实现文件(如tage.scalatourney.scala等),表明BOOM支持可配置的分支预测组件。这些动态预测器利用多位历史模式,大幅提升分支方向预测的准确率。
  • 返回地址栈 (RAS): 对于函数调用和返回指令,BOOM使用RAS来预测返回地址。每遇到Call指令将其下一地址压栈,遇到Ret则从栈弹出预测返回PC。

通过以上机制,BOOM前端可在取指阶段多次重定向指令流:在取指流水线的每个cycle,前端使用分支预测组件判断当前取指包内是否有跳转/分支,如果有则预测其方向和目标PC;若预测为跳转且目标已知,前端会立即将PC切换到预测目标,在下一cycle从新地址取指docs.boom-core.org。这样,即使取指包中有分支,前端也不必等其实际执行就能预先沿预测路径取指,提高并行度。如果后端执行后发现某分支预测错误或有异常发生,则向前端发送Flush信号和正确PC,前端将丢弃错取的指令并从正确PC重新取指docs.boom-core.org。这种前端/后端配合保证了即使发生乱序执行,控制流也能快速恢复。

前端关键的类/模块包括:BoomFrontend(作为LazyModule封装前端总体),其内部BoomFrontendModule连接I-Cache(封装自Rocket的ICache类)、指令TLB、FetchBufferFetchTargetQueue以及分支预测流水线等组件。分支预测部分在源码中由诸如BimodalTagePredictorRAS等类实现,并由BoomBPredictor(见predictor.scala)统一管理。通过参数配置,不同级别的预测器可组合启用,实现性能与硬件成本的折中。

)

解码阶段从前端取指缓冲区获取取指包,对每条指令进行译码,将其翻译成微操作(Micro-Op, uop)并做初步的资源分配检查docs.boom-core.org。BOOM的解码器支持RISC-V标准的RV64GC指令,包括整数、乘除、原子、浮点、CSR等各类操作。其主要功能和流程:

  • 指令译码: 解码器将每条取出的指令位模式译码成内部控制信号(如操作码、源目标寄存器编号、立即数等),生成BOOM内部使用的MicroOp对象。在BOOM源码decode.scala中,DecodeUnit类包含详细的译码查表,将RISC-V指令映射为对应的uop控制信号。
  • 压缩指令展开: 对于16位的RISC-V压缩指令(RVC),BOOM利用Rocket提供的RVCExpander对其进行展开docs.boom-core.org。展开后的指令在微架构上等效为对应的32位非压缩指令。这样,后续流水线不需特意处理压缩指令,简化了实现。
  • 微操作拆分: 某些复杂指令在BOOM中会拆分为多个微操作。例如,RISC-V存储指令会拆成“计算地址”的STA uop和“提供数据”的STD uop(详见LSU部分),AMO原子指令也会拆解成加载、计算和存储多个uop。Decode阶段负责根据指令类型产生适当数量的MicroOps并标记它们的关系(如STA/STD属于同一Store指令)。
  • 资源分配检查: 为保证后续流水线有空间容纳新指令,Decode阶段会检查关键共享资源是否有空闲条目,比如ROB、重命名映射表、Free List、Issue队列、Load/Store队列docs.boom-core.org。如果其中任何一个已满(即当前在飞指令过多),Decode必须停顿(stall),不再从取指缓冲读取新指令,直到资源释放。这样避免了过度发射导致后端溢出。
  • 分支谱系信息: 解码时还会处理分支相关信息,例如每条指令会带有一个分支掩码(Branch Mask),指示当前指令受哪些未决分支的影响。这在BOOM的实现中用于决定分支错预测时需要flush哪些指令。Decode阶段生成并更新这些掩码信息,供Rename阶段及ROB追踪。

实现方面,DecodeUnit(源码decode.scala)包含IO接口DecodeUnitIo(输入取指包,输出解码后的uop序列)。解码输出的每个MicroOp附带了各种控制字段,如操作类型、源/目的寄存器号、立即数、是否为分支或存储等。BOOM的decode支持每周期多发射:若取指包内含多条指令且资源允许,Decode会一次性产生多条uop并并行进入后续Rename阶段。对于RISC-V特有的Fence等内存序列化指令,Decode会特殊处理,在uop中标记并在ROB中进行顺序控制。

总之,Decode阶段将前端提供的原始指令序列转化为BOOM内部的操作序列,并确保后端资源可用,为进入乱序执行做好准备docs.boom-core.org

)

在乱序执行机器中,寄存器重命名是关键环节。BOOM采用了显式重命名(Explicit Renaming)架构,实现了一个统一的物理寄存器文件(PRF),所有的架构寄存器(包括整数和浮点寄存器)在执行前都会映射到PRF中的一个物理寄存器上docs.boom-core.org。重命名阶段的主要功能是消除假相关:通过将指令的源/目的寄存器编号换成物理编号,从而消除写后写(WAW)和写后读(WAR)冲突,仅保留真正的数据依赖(读后写,RAW)docs.boom-core.org

BOOM的重命名阶段在每周期对解码出的每条uop执行如下操作docs.boom-core.org

  • 源寄存器重命名: 对于每个源操作数(逻辑寄存器号),在**重命名映射表 (Rename Map Table)**中查找得到对应的当前物理寄存器号docs.boom-core.org。这样,uop的源操作立即被标记为指向具体的物理寄存器。对于BOOM v4而言,整数寄存器x0-x31和浮点寄存器f0-f31都有各自的映射表条目。
  • 目的寄存器重命名: 如果uop有目的寄存器(写结果),重命名逻辑从自由列表 (Free List)*中分配一个空闲的物理寄存器作为新的目的物理寄存器docs.boom-core.org。同时,在重命名映射表中将该逻辑寄存器更新为新的物理寄存器号。这样后续的指令会看到这个最新映射。在更新前,映射表中原先对应该逻辑寄存器的旧物理寄存器号即成为*“陈旧的目标”(Stale Destination)docs.boom-core.org。BOOM会将这个旧物理寄存器号暂存(通常存入ROB条目),以便等到指令提交后再释放回Free List供重复利用。
  • 分配ROB和队列条目: Rename阶段为每个uop分配一个ROB入口,以及Issue队列、Load/Store队列等所需的条目索引(这些资源已经在Decode时检查过可用)。Rename会将ROB索引、LSQ索引等附加在uop上,供后续阶段使用。
  • 设置Busy位: 对新分配的物理目的寄存器,在**繁忙表 (Busy Table)**中标记为“繁忙”docs.boom-core.org(表示尚无有效数据)。当下游执行单元计算出结果并写回时,会清除相应的繁忙位,表示物理寄存器现在包含有效值。Busy Table通过跟踪物理寄存器是否就绪,辅助Issue队列判断uop依赖是否满足。
  • 分支快照: 为了支持高效的分支误预测恢复,BOOM在Rename阶段对每条分支指令都会快照保存当前的Rename Map表和Free List状态docs.boom-core.orgdocs.boom-core.org。具体来说,每遇到一个分支,Rename Map Table当前内容会复制一份与该分支关联;Free List也会保存当前未分配物理寄存器的列表(或使用一个并行的“已分配列表”来记录此后的新分配)docs.boom-core.org。如果将来该分支发生误预测,恢复机制可以在一个时钟内将Rename Map恢复到分支时的状态,并撤销在此之后的所有物理寄存器分配。这一机制极大提高了分支恢复速度,因为无需逐条回滚指令状态。

Rename阶段高度并行:BOOM设计允许每周期对多条指令同时进行重命名,这需要多端口的映射表和Free List。硬件上,Rename Map Table通常实现为多读多写端口的寄存器阵列,Free List则可用位图或FIFO实现快速分配和回收空闲寄存器。Busy Table往往是一个位向量,长度等于物理寄存器数,其在Rename分配目的寄存器时置1,在写回阶段由执行结果的完成信号清0docs.boom-core.org

BOOM v4的实现中,rename-stage.scala定义了重命名模块的逻辑,包含Map Table (RenameMapTable类)、Free List (RenameFreeList类)和Busy Table (RenameBusyTable类)的具体实现。这些类通过组合,形成Rename阶段的主要子模块。构造参数包括物理寄存器总数、重命名宽度等。主要方法包括分配空闲寄存器、备份/恢复映射表、查询和更新忙碌状态等。通过重命名,所有进入乱序后端的uop都携带物理寄存器标识,从而后端可以完全使用物理寄存器文件进行读写,摆脱架构寄存器数目的限制docs.boom-core.org

) 与派遣阶段 (Dispatch Stage)

一旦指令经过重命名,它们将进入**派遣(Dispatch)阶段,分配到重排序缓冲区(ROB)发射队列(Issue Queue)**中。ROB是乱序处理器维护指令顺序和支持提交的重要结构。BOOM的ROB承担以下角色:

  • 跟踪乱序指令状态: ROB记录所有在飞(in-flight)指令的信息,包括它们的顺序、执行完成状态、异常状态等docs.boom-core.org。ROB的本质是一个循环缓冲区,按照程序顺序排列指令。ROB头指向最早的在飞指令,尾指向最新分派的指令docs.boom-core.org
  • 保证顺序提交: 尽管执行是乱序的,但ROB确保提交(对架构状态的更新)按程序顺序进行,以保持架构上的顺序语义docs.boom-core.org。只有当一个指令到达ROB头且标记为已完成时,才会被提交,更新其结果到架构寄存器/内存,并从ROB移出。这样对软件而言,好像指令是顺序执行的。
  • 异常和分支处理: ROB也负责处理异常和分支错预测。每个ROB项包含一位标识该指令是否产生异常docs.boom-core.org。如果ROB头的指令标记了异常(例如非法指令、存储访问错误等),处理器将在提交该指令时触发异常处理流程:ROB发出流水线flush信号,取消所有未提交的后续指令,并将PC重定向到异常处理入口docs.boom-core.org。对于分支错预测,ROB检测到错预测的提交条件时(通常通过异常标志或专门信号),也会触发前端flush并恢复Rename快照。

BOOM v4的ROB实现细节:

  • 结构和容量: ROB大小(条目数)是参数化的,例如典型配置下可能有numRobEntries=ROB_SZ。为了支持每周期同时分派和提交多条指令,BOOM采用分段(banked)ROB结构docs.boom-core.org。概念上,可视ROB为W行,每行W列的阵列(W为机器宽度,例如发射宽度)。每个周期最多可向ROB写入W条新指令(填满一行)并提交W条已完成指令(从另一行)docs.boom-core.org。这样设计简化了多指令并发操作:取指包中的W条指令占据ROB同一行的多个列,共享一个程序计数器PC(低位由列索引推断),从而减少PC存储开销docs.boom-core.org。当然,如果取指包未满W条(例如遇到分支边界),则该ROB行会有空位,但仍占据一个PC条目docs.boom-core.org
  • ROB项内容: 每个ROB条目存储的信息相对精简,包括:有效位(该entry是否有指令)、完成标志(该指令执行是否完成,即“busy”位)docs.boom-core.org、异常标志(指示是否发生异常)docs.boom-core.org、以及一些需要在提交时更新的少量状态(如分支预测是否正确、存储指令的地址或SC是否成功等)。架构目标寄存器的值一般不存储在ROB中(BOOM采用显式PRF设计,所以结果直接写物理寄存器文件)。ROB更关注指令状态而非数据值。

派遣阶段的流程是:当指令通过Rename后,立即派遣到ROB和Issue队列中docs.boom-core.org。具体而言,BOOM会为每个uop选择一个ROB空闲位置(在ROB尾),并将该uop的一些信息写入ROB条目,同时也将uop送入对应的Issue队列等待执行docs.boom-core.org。派遣操作每周期最多处理与解码/重命名宽度相同数量的uop。若ROB已满或下游Issue队列已满,派遣会暂停,从而也阻止新的指令重命名,直至有空间释放。

值得注意的是,ROB的索引在Rename时就已分配给uop,派遣阶段实际执行将uop写入ROB存储的操作。这在源码中由Rob类完成,其io.enq接口接收派遣的uops写入。ROB对外还提供io.deq用于提交时读出信息,以及异常处理接口等。

BOOM的ROB实现类为Rob(见rob.scala),其构造参数包括ROB大小、发射/提交宽度等。ROB内部通过循环索引和银行划分来管理存储。还有一个RobIo定义了ROB与外部的交互信号(如与Rename、Issue、执行单元、提交单元的接口)github.com。ROB会与执行单元和写回阶段交互,当指令执行完毕会通过完成广播通知ROB清除相应busy位。当ROB头指令busy位为0(完成)且没有等待的异常/分支,需要提交时,ROB触发提交逻辑(见后文)。

总之,ROB与Dispatch阶段一起,起到了连接乱序执行各部分的中枢作用:它记录了指令乱序执行的状态,并最终以正确顺序提交结果,使处理器对编程模型表现为顺序执行docs.boom-core.org

)

发射队列(Issue Queue)是乱序处理器中用于暂存已派遣但尚未执行的微操作的结构。BOOM的Issue单元决定何时从这些等待队列中选择指令发送给执行单元。BOOM v4在Issue单元上的设计具有以下特点:

  • 多队列拆分:BOOM采用分离的Issue队列,根据指令类型划分不同的队列docs.boom-core.org。典型地,BOOM有整数运算队列浮点运算队列访存队列三类docs.boom-core.org。整数ALU指令进入整数Issue Queue,浮点指令进入浮点Issue Queue,访存指令(加载/存储地址计算)进入内存Issue Queue。这样划分可以针对不同指令类型设置不同大小和调度策略的队列,并行处理不同资源的指令,提高效率。
  • 等待依赖:每条进入Issue队列的uop都会有对应的源操作数准备标志。在Rename阶段,若源操作数对应的物理寄存器尚未准备好(Busy Table指示未就绪),则uop在Issue队列中需要等待。Issue队列的每个entry一般包含若干位来跟踪该entry两个(或三个)源操作数是否已准备。初始时未就绪的操作数将标记为“等待”。
  • 唤醒与请求:当执行单元计算完结果写回时,会广播结果的物理寄存器编号(以及结果值)。Issue队列监听这些广播,对比自己的等待源,如果匹配则将对应源标记为“ready”,当一个entry的所有源操作数均准备就绪时,该entry就产生执行请求docs.boom-core.org。具体实现上,每个Issue entry有一个request位,在检测到所有源ready后置1docs.boom-core.org
  • 选择逻辑:每个周期,Issue单元的选择逻辑(Select)*会在所有请求位为1的entry中选择一定数量的uop发送给执行单元执行docs.boom-core.org。典型策略是按照一定优先规则选择,例如*年龄优先(最早进入队列、在等待最久的优先)或者无序**(不考虑年龄,只要准备好即可)。BOOM支持配置不同的选择策略,在文档中提到可以选择Age-ordered Issue QueueUnordered Issue Queuedocs.boom-core.org。年龄优先保证乱序执行仍倾向于按原顺序选取,从而减少饥饿;无序则可能更简单硬件但需要处理starvation。BOOM代码中可能通过issueParams配置不同队列是否采用age-based调度。
  • Issue宽度:Issue选择每周期能选出的uop数量等于处理器的并行执行能力。比如BOOM的整数部分可能有2条ALU和1条内存AGU,那么整数Issue每周期最多可选出3条准备好的uop(分别给两个ALU和一个AGU)。但具体实现中通常每个Issue队列对应一个或多个发射端口,例如整数Issue Queue可能有2个端口(连接两条ALU流水),浮点Issue Queue1个端口。每个端口每周期选出一条uop。因此Issue宽度实际等于所有队列端口数总和。
  • 发射与清除:被选择发射的uop会从Issue队列移除(或标记为无效,以便腾出空间)docs.boom-core.org。BOOM在发射后会将该entry重用或加入空闲列表,供后续派遣新的uop使用。此外,BOOM在设计上也考虑Speculative Issue(推测发射)docs.boom-core.org——即在某些情况下可以不等待所有操作数确定就发射,例如猜测Load会命中缓存并提前发射其依赖算术指令。如果猜测错误则需要回滚。docs.boom-core.org中提到BOOM未来可能考虑此类优化,但截至文档所述版本暂未实现。所以BOOM v4应为当操作数真正就绪才发射的保守调度策略。

BOOM的Issue单元在硬件上对应源码中的IssueUnit类及IssueSlot等。IssueUnit在core.scala中被实例化多次,例如alu_issue_unitmem_issue_unitgithub.com。构造参数IssueParams定义每个队列大小(numEntries)、发射端口数(issueWidth)以及调度策略(是否年龄排序)。IssueUnit内部包含若干IssueSlot(每个slot对应一个entry),以及选择逻辑和优先级编码器来选择请求。BasicDispatcher类用于当指令派遣宽度大于单队列宽度时,将指令分配到多个Issue队列,例如整数指令平均分配到两个并行的整数Issue队列(提高并行度)。

关键交互: Issue单元上游连接Dispatch(接受派遣的uop写入空slot),下游连接执行单元(选择后将uop发送执行)。同时Issue单元通过广播网络连接写回阶段:执行结果的完成会生成wakeup信号广播物理寄存器号github.com。Issue单元将广播与内部等待寄存器比对,匹配则唤醒。BOOM中numIntWakeups等参数定义了广播总线条数。

简而言之,Issue单元是乱序调度的核心,确保当指令的所有依赖满足时,能及时选中执行,同时控制每周期执行单元的吞吐不超过硬件能力docs.boom-core.org。BOOM的多队列和灵活策略设计,使其Issue逻辑可以根据目标频率和负载类型调整,以取得更好性能/面积折衷。

)

当一条uop在Issue阶段被选中发射后,它进入**寄存器读取(Register Read)阶段。在这个阶段,指令在执行运算前需要读取它所需的源操作数值。由于BOOM采用物理寄存器文件(PRF)**架构,每个源操作数对应一个物理寄存器编号,寄存器读取阶段实质是从物理寄存器文件中读取数据。这部分包括:

  • 物理寄存器文件设计: BOOM具有统一的物理寄存器文件,但根据整数和浮点寄存器集的不同,实际上实现为两个物理寄存器文件:一个存放整数/指针数据,另一个存放浮点数据docs.boom-core.org。物理寄存器文件的大小一般大于架构寄存器数,比如RV64有32个整数寄存器,但BOOM可有128个整数物理寄存器(具体数量依配置)。Committed和未提交的值都存放在这个文件中,因而PRF同时扮演了保存架构状态和保存乱序临时状态的角色docs.boom-core.org。BOOM的浮点寄存器文件使用了65位宽度来存储64位浮点数(使用Berkeley Hardfloat库格式,多出的位用于额外精度)docs.boom-core.org
  • 端口配置: 寄存器文件需要提供足够的读/写端口以支撑处理器的并行执行。假设整数部分有N条执行通路需要每条读2个源操作数、浮点部分有M条执行通路每条读2或3个源,则整数PRF需2N个读端口、浮点PRF需2M或3M个读端口,以及对应的写端口用于写回结果。例如,据官方示例,某双发射配置下整数RF需要6个读端口、3个写端口,浮点RF需要3个读端口、2个写端口docs.boom-core.org。BOOM目前采用静态端口分配,即提前划定哪些读端口供哪个执行单元使用,以简化设计docs.boom-core.org。比如端口0和1固定给ALU0使用,端口2和3给Mem单元使用等docs.boom-core.org。这一静态分配避免了读端口在不同单元间争用的仲裁,但可能造成少量端口利用率降低。文档也提到未来可研究动态端口调度以减少端口数量docs.boom-core.org
  • 读出操作: 在寄存器读阶段,每条uop根据其源物理寄存器号,从对应的物理RF读出数据总线。如果前面Busy Table标记该寄存器已准备(即有值),则这里可以正确读到操作数。如果由于某种原因操作数尚未准备好(理论上Issue选择时应已就绪),那么这个uop将无法正确执行,处理器可能需要采取措施(通常不会发生,因为Issue保证了才发射)。
  • 旁路网络 (Bypass Network): 为了减少流水线气泡,BOOM实现了结果旁路。旁路网络将执行单元产生的结果在生成的同周期或下一周期直接转发给正在等待该结果的消费者指令,而不必等结果写回物理寄存器再读出docs.boom-core.org。在BOOM的Pipeline中,ALU等功能单元可能有多级流水,如果不旁路,则一条依赖紧邻上一条的指令需要等好几拍才能拿到结果。BOOM通过在执行单元的流水线各阶段加入旁路MUX,使得例如某条指令在“寄存器读”阶段可以直接获取前一条刚在“执行”阶段产生的结果docs.boom-core.org。文档举例提到,由于ALU流水线被延长匹配FPU延迟,ALU可从这些阶段的任意点旁路给寄存器读阶段docs.boom-core.org。简而言之,如果指令B紧跟指令A且依赖A的结果,A执行完的下一个周期B在RegRead时,通过旁路总线可拿到A的结果,无需等待A写回RF。BOOM的旁路网络支持所有常见的ALU-ALU转发、ALU-AGU转发等,大大降低数据相关带来的等待。
  • 写端口仲裁: 由于多条指令可能同时完成写回,且整数/浮点结果可能需要写不同RF甚至两个RF(例如Load既可能写整数RF又可能写浮点RF),BOOM确保寄存器文件有足够的写端口来容纳峰值写回。同一时刻,一个物理RF的写端口数量 = 提交宽度(因为每周期最多提交这么多写)+ 非同步完成的写回数(比如长延迟单位的额外结果)。BOOM通过设计使得执行单元的最长流水线(如FPU)延迟等于其它单元延迟,某些结果(如ALU结果)可能在流水线中插入空拍以对齐docs.boom-core.org。这样所有单元结果在固定周期写回,减少写端口调度复杂度docs.boom-core.org。在必要情况下,BOOM也可以对写回进行仲裁(如cache返回的数据和ALU结果同时想写整数RF的共用端口时)但通常通过设计避免这种冲突。

BOOM的Register Read在实现上没有单独的模块类,而是作为Issue到Execute阶段过渡的一部分。regfile.scala定义了整数和浮点物理寄存器文件实现,BypassNetwork在流水线各单元中组合实现。总的来说,寄存器读取阶段确保指令在进入执行单元前,其所需的所有源数据已经准备到位,来自于物理RF或旁路,从而可以正确执行后续运算。

)

执行单元是实际执行指令运算的功能模块集合。BOOM将不同类型的运算功能分散到多个执行单元中,并行执行。每个执行单元可视为挂接在Issue发射端口之后的流水线。BOOM典型的执行单元配置包括:

  • 整数算术逻辑单元(ALUs): 负责整型算术和逻辑运算(加减乘、移位、布尔等)以及分支比较等。BOOM通常配置有多个ALU以支持多发射。ALU的执行延迟通常为1个周期(简单运算)或若干周期(乘法等)。v4中ALU管线可能被延长以同步写回时序docs.boom-core.org
  • 分支单元(Branch Unit): 通常集成在一个整数执行单元中,用于处理跳转和分支指令。它计算分支条件并给出实际的下一个PC,如果预测错误则发送纠正信号给前端。BOOM的分支单元还负责调用Return地址栈的更新和Pop等。虽然文档未专门列出,但实现中常将Branch Unit视为特殊ALU功能单元。
  • 乘法/除法单元: 整数乘法在BOOM中可能有专门的流水线(乘法可能做成多周期流水,每拍产出一个结果),整数除法/取模通常是**非流水(unpipelined)**的功能单元,因为除法延迟长且很少用docs.boom-core.org。BOOM可能配置一个共享的整数除法器,每次只能服务一条除法指令,其它除法指令必须在Issue队列等待前面的完成。
  • 加载/存储地址单元(AGU): 负责计算内存访问指令的有效地址。通常称为Address Generation Unit,接受base寄存器和位移立即数,计算出地址送往LSU。在BOOM的架构中,这属于访存执行单元的一部分。AGU通常也是1个周期完成地址计算,并将结果发送给LSU进行缓存访问。
  • 浮点运算单元(FPU): 处理浮点加减乘合并(FMA)、转换、比较等操作。BOOM采用了Berkeley Hardfloat库实现的高性能FPUdocs.boom-core.org。FPU内部有多条流水线:如加法、乘法、乘加可能是多周期流水(通常2-4周期延迟,每周期可发射新操作),浮点除法和开方可能是更长延迟且可能不完全流水(比如使用迭代算法)。BOOM通常配置1个多功能FPU执行单元,可以并行处理不同类型的FP运算,但受限于端口和运算类型搭配(如同时来两个乘加可能一个需等待)。
  • 特殊功能单元: 包括CSR读写单元(处理csrrw等)、内存屏障单元(处理fence等)。CSR指令在BOOM中可能由ALU单元通过与Rocket CSR文件接口协作完成。还有BOOM可以通过Rocket的RoCC接口挂接定制加速器单元,作为执行单元的一种,对于带有customX等自定义指令的操作,会发射到RoCC单元执行。

BOOM将上述执行逻辑按执行端口打包成执行单元(Execution Unit)*的概念。每个Issue端口对应*一个执行单元docs.boom-core.org。例如,在一个双发射配置下:Issue端口0连接执行单元0,该执行单元包含ALU功能和可能的乘法、FPU等;Issue端口1连接执行单元1,包含另一个ALU和Load/Store AGU等docs.boom-core.org。这样,一个执行单元内部可以有多个功能单元(Function Units)**,共享该Issue端口的uop来源docs.boom-core.org。以文档Fig.19为例:执行单元)。BOOM通过这种配置使得每个发射端口的能力被充分利用。例如,端口0主要用于通用和浮点计算,端口1偏重内存和分支等,通过合理分配不同类型uop到不同端口的Issue队列,可减少资源竞争。

执行单元内部通过流水线寄存器将操作逐级推进,最终在写回阶段输出结果。对于流水线化的功能单元(如加法、乘法),执行单元可以每周期都接受一条新指令(如果Issue有供应),不同指令在流水线各级穿行,增加吞吐。对于非流水的功能单元(如整数除法),执行单元在该运算进行期间会阻塞后续同类型指令的进入。BOOM的Issue逻辑会感知这些单元的可用状态,在除法器忙碌时,不会再发射新的除法指令。

BOOM执行单元也处理一些特殊情况:例如对于分支指令,Branch Unit执行后若发现预测错误,会触发整个流水线flush和Recovery;对于SC(Store-Conditional)指令,执行单元需要告知LSU是否成功(是否抢占到了地址),并将结果写回寄存器和标记STQ条目;对于AMO,执行单元需与LSU配合完成读改写操作。总的来说,执行单元负责实际的指令效果计算,并将结果和状态反馈给ROB和后续流程。

源码中,执行单元的组织可见于exu/core.scala里,如实例化了FpPipeline模块(其中含FP执行单元和FP Issue队列)github.com,以及整数执行单元的创建。ExecutionUnits类可能列出和配置不同执行单元及其支持的Operation类型,并生成相应硬件模块实例。关键类包括:ALUUnitMulDivUnitFPUUnitMemAddrCalcUnit等等,这些可能以继承共用的ExecutionUnit特质的方式实现。

综上,BOOM通过多个并行执行单元实现了整数、浮点和内存操作的乱序并行执行docs.boom-core.org。各执行单元内部又集成了多种功能单元以支持丰富的指令类型。它们与Issue队列、物理寄存器文件和ROB共同构成BOOM乱序执行的核心机制。

) 及其队列 (Load/Store Unit, LDQ/STQ)

**加载/存储单元(LSU)是处理器与数据存储系统交互的桥梁。它负责按照程序的内存语义执行乱序处理器中的内存访问,并与数据缓存协同工作。在BOOM中,LSU包括专门的负载队列(LDQ)存储队列(STQ)**来跟踪进行中的内存操作docs.boom-core.org。其主要功能包括:

  • 内存指令微操作划分: 在解码阶段,每条负载/存储指令在LSU中预先分配条目。对于Load指令,会生成一个微操作uopLD;对于Store指令,BOOM拆分为两个微操作:uopSTA(Store Address,用于计算并保存地址)和uopSTD(Store Data,用于准备待存数据)docs.boom-core.org,on the store UOP specifics)。这样的拆分使得存储地址计算和数据提供可以分别乱序执行,提高并行度。docs.boom-core.org,on the store UOP specifics)描述了这两个uop的作用:STA计算地址后写入STQ相应entry的地址域,STD从寄存器读出要存储的数据后写入STQ entry的数据域。
  • 队列分配与有效性: Decode阶段为每个检测到的Load或Store指令在LDQ或STQ中保留一个条目(即使尚未重命名,也要确保队列有空位)docs.boom-core.org。在Rename/Dispatch阶段,uopLD被指派到LDQ的下一个空entry,uopSTA/STD被指派到同一个STQ entry的地址或数据部分。LDQ/STQ条目通常包含多个字段:有效位(valid),地址(addr)及其有效标志,数据(data)及其有效标志(对STQ),执行完成标志(exec),提交标志(commit)等docs.boom-core.org。当Decode保留条目时,条目标记valid,但addr/data无效。随后STA执行完,填入地址并标记地址有效;STD执行完,填入数据并标记数据有效docs.boom-core.orgStore Queue条目还包括committed位,表示该store指令是否已经提交。docs.boom-core.org当store指令在ROB中提交后,对应STQ条目置为committed。
  • 地址计算与存储提交: 执行阶段,Load和Store地址计算由AGU完成:Load的AGU计算完地址后,将地址送入LSU,LSU把地址写入LDQ相应条目;Store的STA类似,将地址写入STQ条目。对于地址为虚地址的,还需通过TLB进行虚实地址转换docs.boom-core.org。BOOM复用Rocket的数据TLB(DTLB),地址计算若遇TLB未命中,会请求PTW页表遍历,在这期间该内存操作需等待。TLB命中则很快得到物理地址。地址进入队列后,LSU会将有效地址用于内存序约束检查缓存访问
  • 内存顺序与旁路转发: 在乱序处理器中,可能出现这样的场景:一个Load本应在程序顺序上晚于某个Store,但乱序执行中Load可能先计算了地址并准备访问内存,而Store地址/数据尚未准备。这就引出存储-加载顺序检查问题。BOOM的LSU采用Store-Load转发和顺序保证机制docs.boom-core.org,on the store UOP specifics)docs.boom-core.org
    • Store-Load转发: 如果有尚未发送到内存的先行Store,其地址与后来的Load地址相同,且Store的数据已准备好,那么Load不必等待真正写回内存,可直接从该Store的数据获得值(旁路转发)。BOOM的LSU包含Searcher逻辑,监视LDQ新进入的地址,与所有未发射/未提交的先行Store地址对比docs.boom-core.org。若匹配且Store已有数据,则将Store数据直接提供给Load(这种情况下Load不访问D$)。如果匹配但Store数据尚未备好,则Load必须等待直到Store的数据到达,再获取后发起docs.boom-core.org
    • 顺序违例检测: 如果一个Load先行执行访问了D$,而之后发现其实在其之前序的某Store地址相同且尚未执行(即Load不该提前),这称为内存顺序违规(Memory Ordering Failure)docs.boom-core.org。BOOM遵循总排序或近似总排序模型,需要处理这种违规。LSU会在Store地址计算出来后,与所有之后发射过的Load地址比对,若发现有地址相同且那些Load已获取了数据,则判定发生了早发射的Load。此时LSU会通知ROB标记相应Load指令导致顺序失败。ROB在处理该异常时会触发flush,将从该Load起的后续指令全部重新执行docs.boom-core.org。同时,可将引起问题的Load重新插回Issue队列或直接等待Store完成后重发访问。通过这种机制,BOOM允许Load在不知道前面Store地址的情况下大胆地先执行,但提供了事后纠错手段,从而获得性能和正确性的折中。
  • 缓存访问与请求控制: 当一个Load指令确定可以安全地访问内存(没有需要等待的先行Store,或已经处理好转发/顺序),LSU就会从LDQ取出其物理地址,向数据缓存发送加载请求docs.boom-core.org。BOOM使用Rocket Chip的非阻塞数据缓存(又称“Hellacache”)docs.boom-core.org。LSU通过一个Cache接口适配层(shim)*与数据缓存交互docs.boom-core.org。该shim在BOOM v4中管理*未完成的Load请求队列,因为BOOM可能乱序发出多个Load并等待返回docs.boom-core.org。如果期间某个Load被判定失效(如顺序违规或指令flush),shim会标记该请求无效,等缓存返回时舍弃结果docs.boom-core.org。数据缓存每周期可接收新请求,3周期后给出返回数据docs.boom-core.org。对于Store,提交阶段ROB头的Store指令被标记committed后,LSU才允许将其对应的STQ entry发送到数据缓存docs.boom-core.org。Rocket的数据缓存采用写通(write-through)/无回送ack策略,对store不显式确认,LSU假定只要没有收到nack即成功docs.boom-core.org。BOOM LSU仍会确保按照程序顺序**逐条发送提交后的Store到缓存,即使后面的Store准备早,也会等待前面的发送以维持内存顺序。
  • 提交与回放: 当Store指令到达ROB头提交时,ROB通知LSU标记STQ该entry为已提交。此后LSU根据缓存空闲情况发送该Store请求给D$(可能与其他已提交Store排队)。当Store写入缓存完成后,LSU从STQ移除该entry。Load指令则在缓存响应回来数据时(且未被取消)写回其物理寄存器,并在ROB中标记完成,使其后续可提交。对于因为顺序问题被flush的Load,会被重新执行;未命中缓存的Load/Store通过MSHR进行Miss处理,从L2/内存获取数据后再完成。

https://docs.boom-core.org/en/latest/sections/load-store-unit.html 图:BOOM v4 加载/存储单元结构示意图。 上图展示了LSU内部组织及数据流。左上为Store队列(STQ),右上为Load队列(LDQ),每个队列条目包含有效(valid)、地址(addr)、数据(data)及各种状态标志(虚拟地址、执行完成、提交等)docs.boom-core.orgdocs.boom-core.org。解码阶段,为每条即将进入乱序的存储或加载指令在STQ或LDQ保留条目(设置valid);执行阶段,AGU计算得到的地址经TLB翻译写入队列(标记addr.valid),Store的数据经STD写入(标记data.valid)。LSU的控制逻辑(中部Controller)监控LDQ新地址与STQ未提交地址的冲突,实现Store-to-Load转发和顺序检查。如检测到顺序违规(order_fail),将引发流水线flush重播Load。对于准备好的内存请求,LSU按顺序将其发送至右下角的L1数据缓存(Data Cache)接口。未命中时通过MSHR进入下级存储(L2);命中则数据直接返回。加载的数据通过旁路送给等待的uop或写物理寄存器,存储则不需要等待确认直接视作完成docs.boom-core.orgdocs.boom-core.org

LSU的实现代码分布在lsu.scaladcache.scalamshrs.scala等文件中。LSU类协调LDQ/STQ及Cache接口。LDQSTQ通常实现为带搜索能力的队列结构。BOOM的LSU确保了乱序执行下内存操作的正确性:既利用乱序和转发提高性能,又通过队列和控制逻辑维护程序的内存访问语义docs.boom-core.orgdocs.boom-core.org

)

BOOM核心并不孤立运行,它通过Rocket Chip的片上网络和Cache子系统与更高层次存储(L2、主存)交互。BOOM充分复用了Rocket Chip成熟的存储基础架构,从而在乱序核心上不必重新实现整个缓存体系docs.boom-core.org。BOOM内存系统关键点:

  • 指令缓存: 如前端部分所述,BOOM使用Rocket的一级指令缓存(I$),这是一种单发射,每拍取指的I-Cache,典型配置下64字节线,4路组相联,Virtually Indexed, Physically Taggeddocs.boom-core.org。指令TLB提供地址转换功能。如果I-Cache未命中,Rocket Chip会通过其非阻塞cache结构发起L2访问。在这种情况下BOOM前端会停顿,直到指令缓存填回所需行。
  • 数据缓存: BOOM集成Rocket Chip的非阻塞数据缓存(NB D$),外号“HellaCache”docs.boom-core.org。HellaCache支持多重Miss并行(通过MSHR机制)和硬件cache一致性协议(TileLink)。BOOM v4中,Data Cache被配置为与Rocket相同的三周期流水线docs.boom-core.org:第一拍接收请求,第二拍访问SRAM,第三拍返回结果docs.boom-core.org。缓存每周期都能接受新请求,理论带宽达到1 个cache line/周期。对于Load请求,如果命中则第三拍即可拿到数据;未命中则占用一个MSHR,等待从L2/内存填充完毕再将数据提供给Load。对于Store请求,Rocket的处理是写通+不确认:即Store写入缓存并通过总线写透,但不会有明确的完成ack,docs.boom-core.org提到“无nack即成功”的策略——LSU只需关心是否收到nack信号,没有nack表示写入成功。
  • BOOM-D$接口Shim: 由于Rocket的缓存最初为顺序CPU设计,BOOM作为乱序核引入了投机执行下的额外需求。为此,BOOM提供了一个适配层(dcache shim),站在LSU和Rocket数据缓存之间docs.boom-core.org。适配层的主要作用有:
    • 维护一个未完成Load请求队列,记录每个发往D$的Load对应的ROB和LDQ信息docs.boom-core.org。如果期间发生Flush(分支错预测或顺序违规)导致某些Load被取消,shim会将这些Load请求标记为无效docs.boom-core.org。当缓存返回数据时,shim查找队列,如果该请求已无效,则丢弃该数据而不发送给LSU/RegFile,以确保投机错误的影响不提交。
    • 协调缓存kill:Rocket缓存协议允许在发出请求后下一个周期取消该请求docs.boom-core.org。BOOM利用这一点,在某些场景如分支错预测flush时,shim可以快速kill最近一个cycle发送的缓存请求,避免浪费带宽在错误路径上docs.boom-core.org。对于更早发出的请求,只能等待返回后丢弃结果。
    • 将Load数据和Store nack转换为BOOM语义下的信号,反馈给LSU控制逻辑。比如若某Load因Cache返回nack(可能内存权限错误等)导致异常,shim需要将此情况通知ROB异常处理。
  • L2及一致性: BOOM通过Rocket Chip的TileLink端口连接片上二级缓存(L2)和系统总线。Rocket Chip的L1 Data Cache具备Cache一致性能力docs.boom-core.org:即使在单核配置下,也能响应外部主机或调试器对内存的访问保持一致docs.boom-core.org。对于BOOM,这意味着比如调试模式下外部可以修改内存,L1会接收到snoop使自己的数据失效。同样,如果BOOM将数据写入cache,其他总线主设备(如DMA引擎)也会得到一致视图。这种一致性机制简化了SoC集成。BOOM本身无特殊处理这一部分,完全由Rocket Chip的缓存一致性协议代理完成。
  • 内存序模型: RISC-V默认内存模型较弱,但提供FENCE指令实现顺序一致化。BOOM通过LSU保证单线程的内存顺序正确(前述顺序失败处理)。对于多线程 memory ordering,因BOOM是单核,这里不涉及MESI之类协议复杂交互,只要遵循TileLink一致性即可。BOOM对FENCE指令的处理是在Decode/ROB级别阻止后续内存操作越过FENCE:即遇到FENCE时,ROB会等待所有先前内存操作提交且完成对外可见后,再允许后续操作执行。此外,对于LR/SC,BOOM LSU和缓存也实现了预留标志,确保在LR到SC中间如果有其他核写入地址则SC失败。

总的来说,BOOM v4的内存系统很好地利用了Rocket Chip已有的基础。在保持乱序执行高性能的同时,通过shim层和LSU逻辑维护了内存访问的正确性和一致性docs.boom-core.orgdocs.boom-core.org。这大大减少了设计复杂度,使开发者更多关注核心乱序逻辑本身。

)

**写回(Writeback)提交(Commit)**是流水线的末段阶段,负责将执行完成的结果更新回处理器状态并对外体现执行次序。

  • 写回阶段: 执行单元在其计算完成时,会将结果写入物理寄存器文件,并通知相关单元依赖已满足。对于1周期执行的ALU指令,计算结果通常在发射后的下一个周期就可以写回;对于多周期的,如乘法可能在发射后第N周期才写回。BOOM安排写回在流水线中固定的位置。例如,当ALU和Load都在执行后第2周期产生成果,则统一在那一拍写回,以简化端口管理。写回时,物理寄存器文件接收写入数据,同时Issue单元接收“wakeup”信号(包含写回的物理寄存器号)用于唤醒等待该值的指令github.com。写回阶段也向ROB发送完成信号,使相应ROB条目标记为非busy(完成)。
  • 提交阶段: 提交是由ROB控制的。当ROB头部的指令标记为已完成且没有异常/分支待处理时,该指令即可提交docs.boom-core.org。提交操作包括:
    1. 将该指令对架构状态的修改正式生效。例如,如果是写寄存器指令,提交意味着这个物理寄存器现在成为架构寄存器的新映射(对于BOOM,因为重命名的关系,架构寄存器状态其实一直在PRF里,只是Rename Map指向了新物理寄存器)。如果是存储指令,提交意味着可以对存储器产生效果(即将commit位写入STQ)。
    2. 从ROB移除该指令的条目(ROB head前进1)。ROB的“提交宽度”一般等于dispatch宽度W,BOOM可在一个cycle内同时提交多达W条已完成且连续的指令docs.boom-core.org。实现上,ROB按行提交,每次如果一整行的指令都完成,则一起提交,从而实现峰值W条/拍的提交吞吐。
    3. 释放该指令占用的物理资源:最重要的是释放它占用的物理寄存器(旧的映射)。BOOM在Rename时保存了每条指令的“stale pdst”(重命名前的旧物理目的寄存器号)docs.boom-core.org在ROB中。当指令提交时,ROB会将此旧物理寄存器归还Free Listdocs.boom-core.org。这样,重命名表已经指向该指令的新物理寄存器,旧的无人引用,可以供后续指令重命名再利用。
    4. 其它清理:如果该指令是分支且曾快照Rename表,则提交时可以丢弃它保存的快照(因为直到提交都未发生错预测,说明分支预测正确,无需恢复);如果有例外(一般不会,因为有异常就不会正常commit),或者SC指令在commit时需要检查是否成功等,也在此处理。对于Store,提交时在STQ标记commit并触发存储写Cache。
  • 异常和分支提交: 当ROB头遇到异常指令或分支未被正确预测的情况,不进行正常提交,而是处理异常/分支恢复docs.boom-core.org。如果是异常(ROB头异常位有效),ROB停止进一步提交,触发流水线flush,将PC设置为异常向量,并交给上层Trap Handler处理异常。乱序机器只在ROB头异常才处理是为了实现精确异常——任何异常只在指令按序到达提交点时才对外表现,后续指令均未对体系结构产生影响,符合顺序语义。如果ROB头是分支指令且确定分支预测错误(例如执行计算得到的目标与FTQ记录不符),同样会flush后续指令并将PC改为正确目标,恢复Rename Map到分支的快照状态,继续执行正确路径docs.boom-core.org。Flush实现上,会清空Fetch Buffer和Pipeline各级无效指令,取消正在执行/发射但未提交的一切操作,使处理器从新PC整齐地继续。BOOM的快照机制保证这种恢复只需一个cycle即可完成,非常高效。

通过ROB的管理,BOOM保证无论乱序执行如何进行,只有当一条指令之前所有指令都已提交且本身完成时才会提交它,从而维护了软件可见的顺序一致性docs.boom-core.org。在提交最后一步,若指令产生了与外界交互(如内存写),这些操作也会在提交后对外部发生。BOOM在提交时也会触发一些调试和统计事件记录,比如提交周期计数、性能事件等(这些在实现中通过PerfCounter记录)。

Commit阶段标志着指令生命周期结束,从取指到提交,BOOM通过上述模块的紧密配合,实现了高效的乱序执行处理。chipyard.readthedocs.io概括地说,BOOM概念上10级流水虽然复杂,但通过合并阶段和预测/快速恢复,使得指令大部分时间都在并行推进,只有必要时才同步排序,从而兼顾了性能与正确性。整套设计对于理解乱序处理器的工作原理和实现方式提供了一个开源且成熟的参考。本文逐模块阐述了BOOM v4的设计要点,希望有助于开发者和架构爱好者深入理解该乱序核心的结构和运行机制。docs.boom-core.orgchipyard.readthedocs.io

CXL 3.0 环境下的操作系统设计:挑战与机遇

1. 引言

数据中心架构正在经历一场深刻的变革,其驱动力源自人工智能 (AI)、机器学习 (ML) 以及大规模数据分析等新兴工作负载的爆炸式增长 1。这些工作负载对计算能力、内存容量和带宽提出了前所未有的要求,推动数据中心向异构计算和分解式基础架构 (Disaggregated Infrastructure) 演进。然而,传统的服务器架构和互连技术,如 PCI Express (PCIe),在满足这些需求方面日益捉襟见肘。CPU 核心数量的增长速度远超每核心内存带宽和容量的增长速度,导致了所谓的“内存墙”问题,即系统性能受到内存访问速度和容量的严重制约 5。此外,PCIe 主要作为一种 I/O 互连,缺乏对缓存一致性的原生支持,限制了 CPU 与加速器、扩展内存之间高效、低延迟的数据共享能力 18。

在此背景下,Compute Express Link (CXL) 应运而生。CXL 是一种基于 PCIe 物理层构建的开放、缓存一致性互连标准,旨在打破传统架构的瓶颈 1。它的核心目标是提供低延迟、高带宽的连接,并在 CPU 和连接的设备(如加速器、内存缓冲器、智能 I/O 设备)之间维护内存一致性,从而实现高效的资源共享、内存扩展、内存池化和内存共享 1。

CXL 3.0 规范的发布标志着 CXL 技术的一个重要里程碑 1。它在前几代 CXL 的基础上,显著增强了 Fabric(结构)能力、交换功能、内存共享机制和点对点通信能力,为构建更大规模、更灵活、更高效的分解式和可组合式系统奠定了基础 1。然而,这些强大的新功能也给操作系统的设计带来了全新的挑战和机遇。操作系统作为硬件资源和应用程序之间的桥梁,必须进行相应的调整和创新,才能充分发挥 CXL 3.0 的潜力。

CXL 的出现,特别是 CXL 3.0 引入的 Fabric、内存共享和 P2P 等特性,不仅仅是对现有 PCIe 总线的简单扩展或性能提升。它预示着计算架构从传统的以处理器为中心向以内存为中心、从节点内资源管理向跨 Fabric 资源管理的根本性转变 3。这种转变要求操作系统设计者重新思考内存管理、资源调度、I/O 处理和安全模型等核心机制,仅仅在现有操作系统上进行修补可能无法充分利用 CXL 带来的优势,甚至可能导致性能瓶颈。因此,操作系统需要进行范式转换,以适应这种新的硬件架构。

本报告旨在深入探讨 CXL 3.0 技术对操作系统设计的具体影响,全面分析操作系统在内存管理、资源调度、I/O 子系统、设备管理和安全机制等方面需要进行的适配和重构。报告将结合 CXL 3.0 的关键特性,分析其带来的性能优势与挑战,梳理当前学术界和工业界在 CXL OS 方面的研究进展和实现状况(特别是在 Linux 内核中的支持),并展望 CXL 及类似 Fabric 技术对未来操作系统架构的长期影响。本报告的结构将围绕用户提出的八个关键问题展开,力求为理解和设计面向 CXL 3.0 的下一代操作系统提供全面而深入的技术洞见。

2. CXL 3.0 技术深度解析

为了理解 CXL 3.0 对操作系统设计的深远影响,首先需要深入了解其关键技术特性及其相较于早期版本的演进。

2.1 从 CXL 1.x/2.0 演进

CXL 标准自 2019 年发布以来经历了快速迭代。

  • CXL 1.x (1.0/1.1): 最初版本主要关注处理器与加速器、内存扩展模块之间的点对点连接 25。它定义了 CXL.io、CXL.cache 和 CXL.mem 三种协议,支持设备缓存主机内存 (Type 1 设备) 或主机访问设备内存 (Type 3 设备),以及两者兼具 (Type 2 设备) 25。CXL 1.1 主要用于内存扩展,允许 CPU 访问连接在 PCIe 插槽上的 CXL 内存设备,缓解服务器内存容量瓶颈 9。此阶段的连接是直接的,不支持交换或池化。

  • CXL 2.0: 于 2020 年发布,引入了关键的单级交换 (Single-Level Switching) 功能 5。这使得单个 CXL 2.0 主机可以连接到交换机下的多个 CXL 1.x/2.0 设备,更重要的是,它实现了内存池化 (Memory Pooling) 11。通过 CXL 交换机和多逻辑设备 (Multi-Logical Devices, MLDs) 功能(一个物理设备可划分为多达 16 个逻辑设备),内存资源可以被多个主机共享(但任一时刻一个逻辑设备只能分配给一个主机)5。CXL 2.0 还引入了全局持久化刷新 (Global Persistent Flush)链路级完整性与数据加密 (Integrity and Data Encryption, IDE) 5。但 CXL 2.0 的带宽仍基于 PCIe 5.0 (32 GT/s),且交换仅限于树状拓扑内的单层交换 5。

  • CXL 3.0: 2022 年发布的 CXL 3.0 是一次重大升级,旨在进一步提升可扩展性、灵活性和资源利用率 1。其关键进步包括:

  • 带宽翻倍: 基于 PCIe 6.0 物理层和 PAM-4 信号,数据速率提升至 64 GT/s,理论带宽翻倍(例如 x16 链路双向原始带宽可达 256 GB/s)1。

  • 零附加延迟: 尽管速率翻倍,但通过优化(如 LOpt Flit 模式)2,其链路层附加延迟相较于 CXL 2.0 保持不变 1。

  • Fabric 能力: 引入了 Fabric 概念,支持多级交换 (Multi-Level Switching) 和非树形拓扑(如 Mesh, Ring, Spine/Leaf),极大地扩展了系统连接的可能性 1。

  • 增强的内存池化与共享: 在 CXL 2.0 池化基础上,增加了真正的内存共享 (Memory Sharing) 功能,允许多个主机通过硬件一致性机制同时、相干地访问同一内存区域 1。

  • 增强的一致性: 引入了新的对称/增强一致性模型,特别是反向失效 (Back-Invalidation, BI) 机制,取代了 CXL 2.0 的 Bias-Based Coherency,提高了设备管理主机内存 (HDM) 的效率和可扩展性 2。

  • 点对点 (Peer-to-Peer, P2P) 通信: 允许 CXL 设备在 Fabric 内直接通信,无需主机 CPU 中转 1。

  • 向后兼容性: CXL 3.0 完全向后兼容 CXL 2.0, 1.1 和 1.0 1。

  • CXL 3.1/3.2 续进: CXL 3.1 (2023年11月) 和 CXL 3.2 (2024年12月) 在 3.0 基础上继续演进。CXL 3.1 重点增强了 Fabric 的可扩展性(如 PBR 扩展)和安全性(引入可信安全协议 Trusted Security Protocol, TSP 用于机密计算)以及内存扩展器的功能(如元数据支持、RAS 增强)22。CXL 3.2 则进一步优化了内存设备的监控和管理(如CXL 热页监控单元 CXL Hot-Page Monitoring Unit, CHMU 用于内存分层)、增强了 OS 和应用的功能性、并扩展了 TSP 安全性 23。这些后续版本虽然超出了本次报告的核心范围(CXL 3.0),但它们指明了 CXL 技术持续发展的方向,对理解 CXL 生态的未来至关重要。

2.2 关键架构特性详解

以下将深入探讨 CXL 3.0 引入的核心架构特性及其对系统设计的影响。

  • Fabric 能力与多级交换:
    CXL 3.0 最具革命性的变化之一是引入了 Fabric 能力 1。这打破了传统 PCIe 基于树状结构的限制,允许构建更灵活、更具扩展性的网络拓扑,如网格 (Mesh)、环形 (Ring)、胖树 (Fat Tree) 或 Spine/Leaf 架构 4。这种灵活性通过多级交换 (Multi-Level Switching) 实现,即 CXL 交换机可以级联,一个交换机可以连接到另一个交换机,而不仅仅是连接到主机和终端设备 1。这与 CXL 2.0 仅支持单层交换形成鲜明对比 5。
    为了管理如此庞大和复杂的 Fabric,CXL 3.0 引入了基于端口的路由 (Port Based Routing, PBR) 机制,这是一种可扩展的寻址方案,理论上最多可支持 4096 个节点 2。这些节点可以是主机 CPU、CXL 加速器(带或不带内存,即 Type 1/2 设备)、CXL 内存设备(Type 3 设备)、全局 Fabric 附加内存 (GFAM) 设备,甚至可以是传统的 PCIe 设备 2。此外,CXL 3.0 允许每个主机根端口连接多个不同类型的设备(Type 1/2/3),进一步增强了拓扑的灵活性 5。多头设备 (Multi-headed Devices) 也是 CXL 3.0 Fabric 的一个特性,允许单个设备(尤其是内存设备)直接连接到多个主机或交换机端口 1。
  • 内存池化与共享:
    CXL 2.0 引入了内存池化的概念,允许将 CXL 连接的内存视为可替代资源,根据需求灵活地分配给不同的主机 2。这主要通过 MLD 实现,一个物理设备可以划分为多个逻辑设备 (LDs),每个 LD 在某一时刻分配给一个主机 5。
    CXL 3.0 在此基础上引入了内存共享 (Memory Sharing) 1。与池化不同,共享允许多个主机同时、相干地访问 CXL 内存的同一区域 2。这是通过 CXL 3.0 的硬件一致性机制(详见下文)来实现的,确保所有主机都能看到最新的数据,无需软件协调 2。
    全局 Fabric 附加内存 (Global Fabric Attached Memory, GFAM) 是 CXL 3.0 实现大规模内存共享和池化的关键设备类型 2。GFAM 设备类似于 Type 3 设备,但它可以被 Fabric 中的多个节点(最多 4095 个)通过 PBR 灵活访问,构成一个大型共享内存池,将内存资源从处理单元中解耦出来 2。
  • 一致性:
    CXL 的核心优势之一是其维护内存一致性的能力 1。这是通过 CXL.cache 和 CXL.mem 协议实现的 4。CXL.cache 允许设备(如 Type 1/2 加速器)一致地缓存主机内存,而 CXL.mem 允许主机一致地访问设备内存(如 Type 2/3 设备的内存)。
    CXL 3.0 引入了增强的/对称的一致性 (Enhanced/Symmetric Coherency) 机制,取代了 CXL 2.0 中效率较低的 Bias-Based Coherency 2。关键在于反向失效 (Back-Invalidation, BI) 协议 2。在 CXL 2.0 中,如果设备修改了其主机管理的内存 (HDM),它无法直接使主机 CPU 缓存中的副本失效,需要复杂的 Bias Flipping 机制。而 CXL 3.0 的 BI 允许 Type 2/3 设备在修改其内存(HDM-D 或 HDM-DB)后,主动通过主机向其他缓存了该数据的设备或主机本身发送失效请求,从而维护一致性 2。这使得设备端可以实现 Snoop Filter,更有效地管理和映射更大容量的 HDM 2。这种对称性也为硬件管理的内存共享奠定了基础 2。
  • 点对点 (P2P) 通信:
    CXL 3.0 实现了设备之间的直接 P2P 通信,数据传输无需经过主机 CPU 中转,从而降低延迟和 CPU 开销 1。这种 P2P 通信发生在 CXL 定义的虚拟层级 (Virtual Hierarchy, VH) 内,VH 是维护一致性域的设备关联集合 5。
    CXL 3.0 利用 CXL.io 协议中的无序 I/O (Unordered I/O, UIO) 流来实现 P2P 访问设备内存 (HDM-DB) 5。UIO 借鉴了 PCIe 的概念,允许在某些情况下放松严格的 PCIe 事务排序规则,以提高性能和实现 P2P 30。当 P2P 访问的目标内存 (HDM-DB) 可能被主机或其他设备缓存时,为了保证 I/O 一致性,目标设备(Type 2/3)会通过 CXL.mem 协议向主机发起 BI 请求,以确保主机端缓存的任何冲突副本失效 5。
  • 带宽与延迟:
    如前所述,CXL 3.0 将链路速率提升至 64 GT/s,基于 PCIe 6.0 PHY 1。为了在更高速度下保持信号完整性,它采用了 PAM-4 调制和前向纠错 (FEC) 2。CXL 3.0 使用 256 字节的 Flit (Flow Control Unit) 格式 2,这与 CXL 1.x/2.0 的 68 字节 Flit 不同。
    关于“零附加延迟”的声明 1,需要强调的是,这指的是与 CXL 2.0 (32 GT/s) 相比,CXL 3.0 (64 GT/s) 在链路层本身没有增加额外的延迟。CXL 3.0 甚至提供了一种延迟优化 (Latency-Optimized, LOpt) 的 Flit 模式,通过将 CRC 校验粒度减半(128 字节)来减少物理层的存储转发开销,可以节省 2-5 ns 的链路延迟,但会牺牲一定的链路效率和错误容忍度 2。然而,这并不意味着 CXL 内存的端到端访问延迟为零或与本地 DRAM 相同。CXL 互连本身、可能的交换机跳数以及 CXL 内存控制器都会引入显著的延迟,通常比本地 DRAM 访问慢数十到数百纳秒 12。因此,尽管 CXL 3.0 提供了更高的带宽,但延迟管理仍然是操作系统面临的关键挑战。

下表总结了 CXL 各主要版本之间的关键特性差异:

表 1: CXL 特性对比 (版本 1.x, 2.0, 3.x)

特性 (Feature) CXL 1.0 / 1.1 (2019) CXL 2.0 (2020) CXL 3.0 (2022) CXL 3.1/3.2 (2023/2024)
最大链路速率 (Max Link Rate) 32 GT/s (PCIe 5.0) 32 GT/s (PCIe 5.0) 64 GT/s (PCIe 6.0) 64 GT/s (PCIe 6.x)
Flit 大小 (Flit Size) 68B 68B 68B & 256B (标准 & LOpt) 68B & 256B
交换级别 (Switching Levels) 不支持 单级 (Single-level) 多级 (Multi-level) 多级
内存池化 (Memory Pooling) 不支持 支持 (通过 MLD) 增强支持 (Fabric, GFAM) 增强支持 (如 DCD)
内存共享 (Memory Sharing) 不支持 不支持 (硬件一致性) 支持 (硬件一致性) 支持
一致性机制 (Coherency Mechanism) CXL.cache/mem CXL.cache/mem (Bias-Based) CXL.cache/mem (增强/对称, BI) 增强/对称, BI
点对点通信 (P2P Communication) 不支持 不支持 支持 (UIO + BI) 增强支持 (如 CXL.mem P2P)
Fabric 拓扑 (Fabric Topology) 点对点 (Point-to-Point) 树形 (Tree-based) 非树形 (Non-tree, Mesh, Ring, etc.) 增强 Fabric (PBR Scale-out)
最大节点数 (Max Nodes) 2 有限 (依赖单级交换机端口) 4096 (通过 PBR) 4096+ (PBR Scale-out)
每根端口多设备 (Multi-Device/Port) 不支持 不支持 支持 (Type 1/2) 支持
链路加密 (Link Encryption - IDE) 不支持 支持 (CXL IDE) 支持 (CXL IDE) 支持 (CXL IDE)
机密计算 (Confidential Computing) 不支持 不支持 不支持 支持 (TSP)
热页监控 (Hot Page Monitoring) 不支持 不支持 不支持 支持 (CHMU)
向后兼容性 (Backward Compatibility) - 兼容 1.x 兼容 2.0, 1.x 兼容 3.0, 2.0, 1.x

数据来源: 1

CXL 3.0 引入的 Fabric、内存共享和 P2P 功能并非孤立存在,而是相互依存、共同构成了其核心价值。Fabric 架构 1 是实现大规模内存池化和共享的基础设施 1,支持灵活的拓扑和多级交换 1。内存共享则依赖于 CXL 3.0 增强的硬件一致性机制(如 BI)来保证数据正确性 2。P2P 通信同样受益于 Fabric 提供的灵活路由,并在访问共享设备内存 (HDM-DB) 时,需要 UIO 与 BI 协同工作以维持一致性 5。这种内在联系意味着操作系统在设计相关管理机制时,必须将这些特性视为一个整体,通盘考虑它们之间的交互和依赖关系,而不能孤立地处理某一个方面。例如,管理内存共享必须理解 Fabric 拓扑和一致性规则,而管理 P2P 则必须考虑 Fabric 路由和潜在的一致性影响。

3. 面向 CXL 3.0 的操作系统内存管理重构

CXL 3.0 带来的内存池化、共享和 Fabric 能力对传统的操作系统内存管理子系统提出了严峻挑战,同时也提供了前所未有的优化机遇。操作系统需要从根本上重新设计其内存管理策略,以适应这种新的内存层级和拓扑结构。

3.1 集成 CXL 内存: NUMA/zNUMA 模型与延迟

操作系统首先需要能够识别和集成 CXL 内存。当前主流的方法是将 CXL 内存设备(尤其是 Type 3 内存扩展器)抽象为无 CPU 的 NUMA (Non-Uniform Memory Access) 节点,通常称为 zNUMA (zero-core NUMA) 或 CPU-less NUMA 节点 27。这种抽象使得 CXL 内存能够相对容易地融入现有的 OS 内存管理框架,应用程序原则上可以像访问远端 NUMA 节点的内存一样访问 CXL 内存 39。

操作系统通过 ACPI (Advanced Configuration and Power Interface) 表来发现和理解 CXL 设备的拓扑结构和内存属性。关键的 ACPI 表包括:

  • SRAT (System Resource Affinity Table): 定义系统物理地址 (SPA) 范围与 NUMA 节点(包括 CXL zNUMA 节点)的亲和性 24。
  • CEDT (CXL Early Discovery Table): 提供 CXL Fabric 拓扑信息,包括 CXL 主机桥 (CHB)、交换机、端口以及它们之间的连接关系,还包含 CXL 固定内存窗口 (CFMW) 结构,描述平台预分配的、可用于映射 CXL 内存的 HPA (Host Physical Address) 窗口及其属性 24。
  • HMAT (Heterogeneous Memory Attribute Table): 提供不同内存域(包括本地 DRAM 和 CXL 内存)的性能特征,如读/写延迟和带宽信息,帮助 OS 做出更明智的内存放置决策 24。

尽管 zNUMA 模型提供了一种集成 CXL 内存的方式,但 CXL 内存的延迟特性与传统 NUMA 节点显著不同。访问 CXL 内存通常会引入比访问本地 DRAM 高得多的延迟。具体延迟值因 CXL 设备类型、连接方式(直连、单级交换、多级交换)、底层内存介质以及系统负载而异。研究和测量表明,CXL 内存访问延迟可能比本地 DRAM 慢 70-90ns(小型池化场景)57,甚至超过 180ns(机架级池化)57,通常是本地 DRAM 延迟的 2-3 倍 46,实测值在 140ns 到 410ns 甚至更高 12。此外,一些研究还观察到 CXL 设备可能存在显著的尾延迟(Tail Latency)问题,即少数访问的延迟远超平均值,这可能对延迟敏感型应用产生严重影响 104。

这种显著的延迟差异使得传统的、主要基于节点距离的 NUMA 管理策略(如 Linux 默认的 NUMA Balancing)在 CXL 环境下效果不佳,甚至可能因为不必要的页面迁移开销而损害性能 27。例如,NUMA Balancing 依赖的 NUMA hinting fault 机制在 CXL 场景下可能失效或效率低下 39。因此,操作系统需要超越简单的 zNUMA 抽象,采用更精细化的方法来管理 CXL 内存。

3.2 高级内存分层策略

鉴于 CXL 内存与本地 DRAM 之间显著的性能差异,内存分层 (Memory Tiering) 成为管理 CXL 内存的关键策略 12。其核心思想是将访问频繁的“热”数据放置在快速的本地 DRAM 层,而将访问较少的“冷”数据放置在容量更大但速度较慢的 CXL 内存层,从而在扩展内存容量的同时,最大限度地减少对应用程序性能的影响 12。

实现高效的内存分层需要解决两个核心问题:准确识别热/冷数据和低开销地迁移数据。

  • 热度识别 (Profiling):

  • 传统方法:许多早期或简单的分层系统依赖基于近时性 (Recency-based) 的方法,例如利用页表中的访问位 (Accessed Bit)。但这种方法不够准确,因为最近访问过的页面不一定是真正的热页面,尤其是在本地 DRAM 容量有限的情况下,可能导致错误的驱逐决策 120。

  • 改进方法:基于频率 (Frequency-based) 的方法能更准确地识别热页,但传统的频率统计(如为每个页面维护计数器)会带来巨大的内存和运行时开销,尤其是在管理 TB 级内存时 120。

  • OS 级技术:Linux 内核提供了一些机制,如定期扫描 PTE (Page Table Entry) 的访问位或利用 NUMA Hint Faults 进行采样,但这些方法开销较大,且可能缺乏对 LLC (Last-Level Cache) 未命中的感知 27。使用硬件性能计数器 (如通过 perf 工具或 Intel TMA) 可以提供更精确的 CPU 行为信息,但将其直接映射到页面热度仍有挑战 100。

  • 硬件辅助:为了克服 OS 级分析的开销和精度限制,研究人员提出了将分析功能卸载到硬件的方案。例如,NeoMem 项目提出在 CXL 设备控制器端集成 NeoProf 单元,直接监控对 CXL 内存的访问并向 OS 提供页面热度统计 96。CXL 3.2 规范也引入了 CHMU (CXL Hot-Page Monitoring Unit),旨在标准化设备端的热页跟踪能力,为 OS 提供更高效的热度信息 23。FreqTier 则采用概率数据结构(Counting Bloom Filter)在软件层面以较低开销近似跟踪访问频率 120。

  • 页面迁移 (Migration):

  • 基本操作:内存分层涉及将页面在不同层级之间移动。提升 (Promotion) 指将热页从慢速层(CXL)移到快速层(本地 DRAM),降级 (Demotion) 指将冷页从快速层移到慢速层 27。

  • 开销与挑战:页面迁移本身是有开销的,涉及页表解映射、数据拷贝和重映射等步骤 119。频繁或不当的迁移可能导致内存颠簸 (Thrashing),反而降低性能 101。

  • 优化技术:为了减少迁移开销,研究者提出了一些优化方法。异步迁移 (Asynchronous Migration) 将迁移操作移出应用程序的关键执行路径 119。事务性迁移 (Transactional Migration) 确保迁移过程的原子性 119。页面影印 (Page Shadowing)(如 NOMAD 系统采用)在将页面从慢速层提升到快速层后,在慢速层保留一个副本,当快速层内存压力大需要降级页面时,可以直接使用影子副本,避免了数据拷贝的开销 119。FreqTier 则根据应用的内存访问行为动态调整分层操作的强度,减少不必要的迁移流量和对应用的干扰 120。

  • 具体实现与研究:

  • TPP (Transparent Page Placement): 由 Meta 开发并部分合入 Linux 内核 (v5.18+),TPP 是一种 OS 级的透明页面放置机制 27。它采用轻量级的回收机制主动将冷页降级到 CXL 内存,为新分配(通常是热的)页面在本地 DRAM 中预留空间 (Headroom)。同时,它能快速地将误判或变热的页面从 CXL 内存提升回本地 DRAM,并尽量减少采样开销和不必要的迁移 27。

  • FreqTier: 采用基于硬件计数器和 Counting Bloom Filter 的频率分析方法,以低内存开销实现高精度的热页识别,并动态调整迁移强度 120。

  • NeoMem: 提出硬件/软件协同设计,在 CXL 设备控制器侧实现 NeoProf 硬件分析单元,为 OS 提供精确、低开销的热度信息 96。

  • NOMAD: 提出非独占式内存分层 (Non-exclusive Memory Tiering) 概念,通过页面影印和事务性迁移来缓解内存颠簸和迁移开销 119。

  • DAMON (Data Access MONitor): Linux 内核中的一个通用数据访问监控框架,可用于内存管理优化。近期有补丁提议为其增加 DAMOS_MIGRATE_HOT/COLD 操作,以支持基于 DAMON 的内存分层 130。

  • Intel Flat Memory Mode: 一种硬件管理的内存分层方案,在内存控制器 (MC) 中以缓存行粒度透明地管理本地 DRAM 和 CXL 内存之间的数据放置,对 OS 透明 24。虽然对 OS 简化,但缺乏灵活性,且可能在多租户环境中引发争用问题 105。

3.3 虚拟内存与页表影响

CXL 引入的异构内存层级也对虚拟内存系统和页表管理提出了新的挑战。

  • 页表放置: 在传统的 NUMA 系统或包含 NVMM (Non-Volatile Main Memory) 的系统中,已经观察到如果页表自身的页面(Page Table Pages, PTPs)被放置在较慢的内存层,会导致页表遍历(Page Walk)延迟显著增加,从而影响应用程序性能,尤其是对于 TLB (Translation Lookaside Buffer) 未命中率高的大内存应用 131。CXL 内存的延迟特性使得这个问题更加突出。如果 OS 不加区分地将 PTPs 分配到 CXL 内存,将严重拖慢地址翻译过程。

  • 解决方案: 需要 OS 采用显式的页表放置策略,将 PTPs 与普通数据页面区别对待,并优先将 PTPs 放置在最快的内存层(通常是本地 DRAM)131。即使在本地 DRAM 压力较大时,也应避免将 PTPs 驱逐到 CXL 内存,或者在 DRAM 空间可用时尽快将其迁回。研究工作如 Mitosis 提出了跨 NUMA 节点透明地复制和迁移页表的方法,以缓解页表遍历的 NUMA 效应,类似思想可应用于 CXL 环境 131。

  • CXL 共享内存与虚拟内存: CXL 3.0 引入的硬件一致性内存共享 2(或基于 CXL 2.0 池化内存的软件一致性共享 33)允许不同主机或同一主机上的不同进程映射和访问同一块物理内存区域。这对虚拟内存系统提出了新的要求:

  • 跨域映射管理: OS 需要能够为不同主机/进程建立到同一 CXL 共享物理内存区域的虚拟地址映射。

  • 一致性维护: 虽然 CXL 3.0 提供了硬件一致性,OS 仍需确保虚拟内存层面的映射和权限管理与底层硬件一致性状态协同工作。

  • 地址空间管理: 在共享内存环境中,需要仔细管理虚拟地址空间,避免冲突,并提供有效的同步原语(可能利用 CXL 的原子操作支持)33。

3.4 OS 机制:CXL 内存池化与共享

操作系统需要提供明确的机制来支持和管理 CXL 的内存池化和共享功能。

  • 内存池化 (CXL 2.0+):

  • 资源发现与分配: OS 需要与 Fabric Manager (FM) 交互,发现可用的内存池资源,并根据应用程序或虚拟机的需求请求分配内存 5。这涉及到理解 MLD 的概念,并将分配到的逻辑设备内存集成到 OS 的内存视图中(通常作为 zNUMA 节点)。

  • 动态容量管理: CXL 3.0/3.1 引入了动态容量设备 (Dynamic Capacity Devices, DCDs),允许在运行时动态增减设备的可用容量,而无需重启或重新配置 79。OS 需要与 FM/Orchestrator 协同,平滑地处理这种容量变化,调整内存映射和管理结构。

  • 高效分配/释放: OS 需要提供高效的机制来管理从池中分配到的内存,并在不再需要时将其释放回池中,以实现资源的高效利用 49。

  • 内存共享 (CXL 3.0+):

  • 共享区域映射: OS 需要提供接口,允许进程或跨主机的应用程序映射到指定的 CXL 共享物理内存区域。

  • 利用硬件一致性: OS 应利用 CXL 3.0 提供的硬件一致性机制(如 Back-Invalidation)来简化共享内存编程模型,避免复杂的软件一致性协议 2。

  • 与 CXL 2.0 对比: 需要区分 CXL 3.0 硬件一致性共享与基于 CXL 2.0 池化内存实现的软件一致性共享 33。后者需要 OS 或应用程序承担更多的一致性维护责任。

  • 接口设计: OS 可以考虑扩展现有的 IPC 共享内存接口(如 System V SHM、POSIX SHM)或借鉴 HPC 中 OpenSHMEM 等模型的思想,来提供对 CXL 共享内存的访问 33。

  • 性能与一致性权衡: 硬件一致性虽然简化了编程,但其协议开销(如 BI 流量、Snoop Filter 查找)可能成为性能瓶颈,尤其是在大规模共享或高争用场景下 73。

zNUMA 抽象虽然为 CXL 内存的初步集成提供了便利途径,但其粒度过于粗糙,无法充分反映 CXL 内存系统的复杂性和异构性 27。CXL 内存的实际性能(延迟、带宽、尾延迟)受到拓扑结构(直连、交换级数)、设备类型(ASIC/FPGA)、底层介质甚至工作负载模式的显著影响 39。简单的 NUMA 距离无法捕捉这些细微差别,导致基于此的默认策略(如 Linux NUMA Balancing)效果不佳 27。为了做出真正有效的内存放置和迁移决策,操作系统需要超越基本的 NUMA 模型,获取并利用更细粒度的信息,例如通过 ACPI HMAT 获取的性能数据、通过 CXL CDAT (Coherent Device Attribute Table) 获取的设备特征 67,或者通过 CXL 3.2 CHMU 等硬件监控单元获取的实时访问统计 23。这意味着 OS 需要更丰富的接口和内部模型来理解 CXL Fabric 的拓扑结构和各部分的性能特征。

有效的 CXL 内存分层不仅仅是简单地将冷页移到慢速层。为了保证对延迟敏感的应用或具有突发内存分配模式的工作负载的性能,主动管理快速层(本地 DRAM)至关重要。仅仅在内存压力出现时被动地降级页面可能导致新的、很可能是热的页面分配被迫进入慢速的 CXL 层,从而造成性能损失 27。Meta 的 TPP 设计明确强调了需要主动进行页面降级,以在快速层中保持足够的空闲空间(Headroom)来满足新的分配需求 27。NOMAD 系统也致力于将迁移操作移出关键路径 119。因此,操作系统分层算法应包含主动维护快速层空闲空间的机制,例如通过预测未来的分配需求,或者对较冷的页面采用更积极的降级策略,同时需要仔细权衡迁移成本。

CXL 3.0 提供的硬件一致性内存共享 2 极大地简化了多主机或多进程共享数据的编程模型 49。然而,这种便利性并非没有代价。底层的硬件一致性协议,特别是 Back-Invalidation 和 Snoop Filter,会引入额外的通信开销和潜在的可扩展性瓶颈,尤其是在大规模共享或高争用情况下 73。研究(如 CtXnL 73)表明,对于某些类型的数据访问(例如事务处理中的元数据访问),严格的硬件一致性可能是“过度设计 (overkill)”。在这种情况下,强制使用硬件一致性可能会牺牲性能。因此,未来的操作系统可能需要提供更灵活的一致性管理选项,例如允许应用程序为特定的共享内存区域选择性地放松一致性保证,或者提供接口让应用程序或中间件能够显式地管理一致性(类似于软件 DSM 的方式),从而在易用性和性能之间找到更好的平衡点,而不是采用“一刀切”的硬件一致性模型。

4. CXL Fabric 中的 OS 资源管理与调度

CXL 3.0 引入的 Fabric 架构将资源管理的范围从单个服务器节点扩展到了跨越多个节点、交换机和设备的互连结构。这要求操作系统具备 Fabric 感知能力,并采用新的资源管理和调度策略。

4.1 Fabric 感知 OS: 与 Fabric Manager 交互

CXL Fabric 的核心管理实体是 Fabric Manager (FM) 5。FM 是一个逻辑概念,负责配置 CXL 交换机、分配池化和共享资源(如将 MLD 的逻辑设备分配给主机、绑定交换机端口到主机的虚拟层级 VH)、管理设备热插拔、设置安全策略等高级系统操作 5。FM 的具体实现形式多样,可以嵌入在交换机固件中、作为主机上运行的管理软件,或集成在基板管理控制器 (BMC) 中 6。

操作系统需要与 FM 进行交互以实现对 Fabric 资源的有效管理。这种交互包括:

  • 发现与拓扑感知: OS 需要能够发现 FM 的存在,并从 FM 获取 Fabric 的拓扑结构信息(哪些设备连接在哪些交换机端口,交换机如何互连等),以及资源的可用状态。
  • 资源请求与释放: 当 OS 需要为应用程序或虚拟机分配来自 Fabric 的资源(如 CXL 内存池中的内存、共享的加速器)时,它需要向 FM 发出请求。同样,当资源不再需要时,OS 应通知 FM 以便释放。
  • 动态配置管理: 对于支持动态容量的设备 (DCDs) 79,OS 需要与 FM/Orchestrator 协同处理容量变化事件。OS 也需要通过 FM 来管理 Fabric 中设备的热插拔和复位等生命周期事件 137。

OS 与 FM 之间的通信接口是实现 Fabric 感知 OS 的关键。CXL 规范定义了 FM API,可以通过组件命令接口 (Component Command Interface, CCI) 进行访问,而 CCI 可以通过 Mailbox (内存映射 I/O) 或 MCTP (Management Component Transport Protocol)(通常用于带外管理,如通过 I2C 或 VDM)传输 6。对于带内管理,OS 通常使用 Mailbox CCI。此外,一些外部 FM 实现可能提供 REST API 或 GUI 接口 134。

然而,当前 OS-FM 交互面临的主要挑战是缺乏统一且健壮的标准接口 11。不同的 FM 实现可能采用不同的接口和协议,导致 OS 需要适配多种机制,增加了复杂性并可能导致厂商锁定。此外,如何清晰地界定 OS 资源管理与 FM/Orchestrator 资源编排的职责边界,如何确保 OS 视图与 Fabric 实际状态的一致性,以及如何处理 FM 故障或不可用的情况,都是需要解决的关键问题 64。一个标准化的、功能完善的 OS-FM API 对于 CXL Fabric 的广泛应用至关重要,它需要覆盖资源发现、请求、配置、状态监控和事件通知等各个方面。

4.2 高级调度算法

传统的操作系统调度器主要关注单个节点内的 CPU 和内存资源,其决策基于本地 NUMA 拓扑和进程/线程状态。然而,在 CXL Fabric 环境中,内存和加速器等资源分布在整个 Fabric 中,访问延迟和带宽因路径和设备的不同而异。因此,需要开发新的 Fabric 感知调度算法 27。

  • 延迟感知调度 (Latency-Aware Scheduling): 调度器应将任务(进程或线程)放置在能够以最低延迟访问其所需内存(无论是本地 DRAM 还是 CXL 内存池/共享区)和加速器的计算节点上 84。这需要调度器了解 Fabric 拓扑(例如,访问某个 CXL 内存需要经过多少跳交换机)84 并获取不同路径的延迟信息(可能通过 HMAT 或 FM 获取)104。仅仅依赖静态的 NUMA 距离是不够的。
  • 带宽感知调度 (Bandwidth-Aware Scheduling): 调度器需要考虑 CXL 链路、交换机端口和内存设备本身的带宽限制 26。它应避免将过多带宽密集型任务调度到会争用同一链路或设备的位置,导致拥塞。对于需要大量 P2P 通信的任务,调度器应尝试将它们放置在 Fabric 中靠近的位置,或选择带宽充足的路径。研究如 Tiresias 提出了利用 Intel RDT 等技术为不同类型的工作负载(延迟敏感型 vs. 吞吐量敏感型)提供差异化的内存带宽分配,并利用 CXL 内存作为补充带宽资源 124。
  • 局部性优化 (Locality Optimization): CXL 的核心优势之一是缓存一致性,它允许计算单元(CPU 或加速器)缓存远程数据,减少数据移动。调度器应利用这一点,将任务调度到尽可能靠近其工作集数据(无论数据在本地 DRAM、CXL 内存池还是共享区域)或所需加速器的位置 27。例如,Apta 系统为 FaaS 设计了感知对象位置的调度策略 144,CXL-ANNS 则根据图数据的访问模式进行调度和预取 148。
  • 与内存分层集成: 调度决策应与内存分层策略紧密协调 27。例如,当内存分层系统将一个任务的热页面提升到某个节点的本地 DRAM 时,调度器应考虑将该任务也迁移到该节点以获得最佳性能。反之,如果一个任务被调度到某个节点,内存管理器应优先将该任务的热数据迁移到该节点的快速内存层。

一些研究项目已经开始探索这些方向。微软的 Pond 项目使用机器学习模型来预测 VM 的延迟敏感性和内存使用模式,以决定将其放置在本地 DRAM 还是 CXL 池化内存上,并分配适当的内存比例 57。EDM 提出了一种网络内调度机制,用于优化分解式内存系统的消息完成时间 143。这些研究表明,未来的调度器需要更智能,能够利用 Fabric 的拓扑信息、实时的性能遥测数据(可能来自 FM 或 CDAT)以及对工作负载特征的理解(可能通过在线分析或离线训练的模型)来做出复杂的放置决策。

4.3 通过 CXL 管理异构加速器 (Type 1/2 设备)

CXL 不仅用于内存扩展和池化 (Type 3 设备),也为连接和管理异构加速器(如 GPU、FPGA、DPU、ASIC 等 Type 1 和 Type 2 设备)提供了统一的、高性能的接口 4。

操作系统在通过 CXL 管理这些加速器时扮演着关键角色:

  • 发现与配置: 使用 CXL.io 协议发现连接的 Type 1/2 设备,读取其能力,并通过 Mailbox CCI 或其他机制进行配置 24。加载相应的设备驱动程序。

  • 内存管理:

  • 对于 Type 2 设备,OS 需要管理其设备自带的内存 (HDM-D 或 HDM-DB),通过 CXL.mem 协议将其映射到主机的物理地址空间,并可能参与内存分层或作为 P2P 通信的目标 2。

  • 利用 CXL.cache 协议,OS 可以使 Type 1/2 设备能够一致地访问和缓存主机内存,减少数据拷贝开销,实现主机与加速器之间更紧密的协作 3。

  • Fabric 中的资源分配: 在 CXL Fabric 环境中,加速器也可能被池化并通过交换机连接。OS 需要与 FM 交互,将特定的加速器资源动态地分配给需要它们的主机或任务 6。CXL 3.0 支持在单个根端口下连接多个 Type 1/2 设备,增加了连接密度和灵活性,也对 OS 的管理能力提出了更高要求 5。

  • 调度考量: OS 调度器需要将计算任务与其所需的、可能分布在 Fabric 不同位置的加速器进行协同调度。同时,需要优化数据放置策略,例如,是将输入数据放在主机内存中让加速器通过 CXL.cache 访问,还是直接将数据加载到加速器的 HDM 中(如果可用且性能更优)。

CXL Fabric 环境下的延迟感知调度面临比传统 NUMA 感知调度更大的复杂性。简单的物理距离或 NUMA 节点 ID 不再能准确反映真实的访问成本。调度器必须综合考虑静态拓扑(如交换机跳数 38)和动态因素,如链路当前的负载和拥塞情况、目标 CXL 设备的类型和内部状态、以及 CXL 协议本身(尤其是一致性协议)带来的开销 5。CXL 内存和设备的性能本身也可能存在显著差异 89。因此,未来的 OS 调度器不能再依赖简化的模型,而需要更强大的感知能力,能够获取并利用详细的 CXL Fabric 拓扑信息、实时的性能遥测数据(可能通过 CDAT 67 或 FM 135 提供),并结合对工作负载延迟敏感性的理解(可能通过在线分析或预测模型 57),才能做出有效的、适应动态 Fabric 环境的调度决策。

5. 适配 OS I/O 子系统与设备管理

CXL 3.0 的 Fabric 拓扑和 P2P 通信能力对操作系统的 I/O 子系统和设备管理框架提出了新的要求。OS 需要能够发现、枚举、配置和管理在复杂、动态拓扑中的 CXL 设备,并支持新的通信模式。

5.1 复杂拓扑中的设备发现、枚举与配置

CXL 设备的发现和初始配置在很大程度上依赖于 CXL.io 协议,该协议基于并扩展了 PCIe 的机制 5。OS 通过标准的 PCIe 枚举流程扫描总线,并通过设备类代码 (Class Code)(例如 CXL 内存设备有特定类代码)和 CXL 定义的 DVSEC (Designated Vendor-Specific Extended Capabilities) 来识别 CXL 设备及其能力 24。需要注意的是,CXL 1.1 设备通常被枚举为根联合体集成端点 (RCiEP),而 CXL 2.0 及更高版本的设备则被枚举为标准的 PCIe 端点,这影响了 OS 如何访问其配置空间和寄存器 67。

CXL 3.0 的 Fabric 架构给设备枚举带来了新的复杂性。在包含多级交换机的非树形拓扑中,OS 可能无法直接通过传统的 PCIe 扫描发现所有连接的设备 4。Fabric Manager (FM) 在这里扮演了重要角色,它可以提供 Fabric 的拓扑信息给 OS,帮助 OS 构建完整的设备视图 11。此外,大规模 Fabric 需要可扩展的寻址机制,PBR (Port Based Routing) 因此被引入,允许 Fabric 中的任意节点(最多 4096 个)相互寻址 2。OS 需要能够理解和使用 PBR 地址来进行设备定位和通信。

在 Linux 中,用户可以使用 lspci、cxl list 等命令或检查 /sys 文件系统来查看 CXL 设备和拓扑信息 24。内核中的 CXL 子系统(包含 cxl_core, cxl_pci, cxl_acpi 等模块)负责解析 ACPI 表(特别是 CEDT),发现 CXL 组件(主机桥、根端口、交换机、端点),并构建内核内部的拓扑表示 24。cxl_test 内核模块可用于在没有真实硬件的情况下仿真 CXL 拓扑以供测试 137。近期针对 AMD Zen5 平台的补丁还涉及处理 CXL 地址转换(HPA 到 SPA)的问题 155。

5.2 管理 CXL.io 与控制接口 (Mailbox CCI)

CXL.io 协议不仅用于初始发现和配置,也承载着运行时的控制和管理通信 24。OS 通过 CXL.io 发送非一致性加载/存储 (load/store) 命令来访问 CXL 设备的寄存器、报告错误以及使用 Mailbox 机制进行更复杂的交互 5。

组件命令接口 (Component Command Interface, CCI) 是 CXL 规范定义的用于管理 CXL 组件(设备、交换机等)的标准接口 6。CCI 定义了一系列命令集(如通用命令、内存设备命令、FM API 命令等)6。CCI 可以通过两种传输机制实现:

  1. Mailbox CCI: 基于内存映射 I/O (MMIO) 的寄存器接口,通常位于设备的 PCIe BAR 空间中。OS 主要通过这种方式进行带内管理 6。Mailbox 通常分为 Primary 和 Secondary 两种,具有命令/状态寄存器、载荷寄存器,并可选支持中断 (MSI/MSI-X) 通知完成。对于耗时操作,CCI 支持后台命令 (Background Operations) 机制 6。
  2. MCTP-based CCI: 将 CCI 命令封装在 MCTP 消息中,通过 I2C、VDM (Vendor Defined Message) 等带外通道传输。这主要用于 BMC 或外部 Fabric Manager 进行带外管理 6。

Linux CXL 子系统提供了对 Mailbox CCI 的支持。cxl_pci 驱动负责枚举设备的 Mailbox 寄存器接口,并将其注册到 cxl_core 137。内核提供 ioctl 接口供用户空间工具(如 cxl-cli 或使用 libcxlmi 库的应用)发送 CCI 命令 113。为了支持厂商特定的功能或固件更新等操作,内核还提供了 CONFIG_CXL_MEM_RAW_COMMANDS 选项以允许发送未经内核校验的原始 Mailbox 命令 94。QEMU 也提供了对 CXL Mailbox 的仿真支持 112。

5.3 启用和管理点对点 I/O (UIO)

CXL 3.0 的 P2P 通信能力允许设备直接访问 Fabric 中其他设备的内存(特别是 HDM-DB),这依赖于 Unordered I/O (UIO) 机制 5。UIO 允许 P2P 流量在某些条件下绕过严格的 PCIe 排序规则,从而可能获得更好的性能 30。

操作系统的角色包括:

  • 能力协商与启用: OS 需要识别设备和路径是否支持 UIO,并进行必要的配置以启用该功能。
  • 路由配置: OS(可能需要与 FM 协作)需要配置 Fabric 中的交换机和端口,以允许 UIO 流量在 P2P 端点之间正确路由(可能使用 PBR)2。
  • 一致性管理: 如前所述,当 UIO 用于访问可能被缓存的 HDM-DB 时,OS 需要确保一致性得到维护。这可能涉及到协调目标设备发起的 Back-Invalidation (BI) 流程 5。
  • 接口提供: OS 需要向上层(应用程序或驱动程序)提供发起和管理 P2P UIO 传输的接口。

目前,UIO P2P 仍然面临一些挑战。CXL 规范本身对 UIO P2P 访问的保护机制规定不足 35。在复杂的 Fabric 中管理 P2P 路由和一致性可能非常复杂。从 Linux 内核的 CXL 成熟度图来看,对 Fabric 和 GFAM 的支持仍处于早期阶段 ( 分),意味着对 UIO P2P 的完整支持可能尚未实现 98。此外,UIO 放松的排序规则可能给 OS 或应用程序带来额外的复杂性,需要确保数据一致性和正确性 30。

CXL 引入的多协议(.io,.cache,.mem)、多设备类型(Type 1/2/3, MLD, GFAM)、动态 Fabric 拓扑以及新的管理接口(CCI, FM API)5 使得 CXL 设备管理比传统的 PCIe 设备管理复杂得多。简单的基于树状总线的枚举和配置模型不再适用。操作系统需要一个更加复杂和动态的设备模型,能够理解 Fabric 拓扑,处理不同协议和设备类型的交互,并与 Fabric Manager 协同工作。Linux CXL 子系统的设计 [24, S_

Works cited

  1. CXL Consortium releases Compute Express Link 3.0 specification to expand fabric capabilities and management, accessed April 19, 2025, https://computeexpresslink.org/wp-content/uploads/2024/01/CXL_3.0-Specification-Release_FINAL-1.pdf
  2. Compute Express Link 3.0 - Design And Reuse, accessed April 19, 2025, https://www.design-reuse.com/articles/52865/compute-express-link-3-0.html
  3. What is Compute Express Link (CXL) 3.0? - Synopsys, accessed April 19, 2025, https://www.synopsys.com/blogs/chip-design/what-is-compute-express-link-3.html
  4. Understanding How CXL 3.0 Links the Data Center Fabric - Industry Articles, accessed April 19, 2025, https://www.allaboutcircuits.com/industry-articles/understanding-how-cxl-3.0-links-the-data-center-fabric/
  5. CXL 3.0: Enabling composable systems with expanded fabric capabilities - Compute Express Link, accessed April 19, 2025, https://computeexpresslink.org/wp-content/uploads/2023/12/CXL_3.0-Webinar_FINAL.pdf
  6. CXL Fabric Management - Compute Express Link, accessed April 19, 2025, https://computeexpresslink.org/wp-content/uploads/2023/12/20220322_CXL_FM_Webinar_Final.pdf
  7. CXL – GAMECHANGER FOR THE DATA CENTER - Dell Learning, accessed April 19, 2025, https://learning.dell.com/content/dam/dell-emc/documents/en-us/2023KS_Jaiswal-CXL_Gamechanger_for_the_Data_Center.pdf
  8. CXL 3.0 and the Future of AI Data Centers | Keysight Blogs, accessed April 19, 2025, https://www.keysight.com/blogs/en/inds/ai/cxl-3-0-and-the-future-of-ai-data-centers
  9. Orchestrating memory disaggregation with Compute Express Link (CXL) - Intel, accessed April 19, 2025, https://cdrdv2-public.intel.com/817889/omdia%E2%80%93orchestrating-memory-disaggregation-cxl-ebook.pdf
  10. Reimagining the Future of Data Computing with Compute Express Link (CXL) Tech-Enabled Interconnects from Amphenol, accessed April 19, 2025, https://www.amphenol-cs.com/connect/reimagining-the-future-of-data-computing-with-cxl-tech-enabled-interconnect.html
  11. Introducing the CXL 3.0 Specification - SNIA SDC 2022, accessed April 19, 2025, https://www.sniadeveloper.org/sites/default/files/SDC/2022/pdfs/SNIA-SDC22-Agarwal-CXL-3.0-Specification.pdf
  12. CXL Memory Expansion: A Closer Look on Actual Platform - Micron Technology, accessed April 19, 2025, https://www.micron.com/content/dam/micron/global/public/products/white-paper/cxl-memory-expansion-a-close-look-on-actual-platform.pdf
  13. Compute Express Link(CXL), the next generation interconnect, accessed April 19, 2025, https://www.fujitsu.com/jp/documents/products/software/os/linux/catalog/NVMSA_CXL_overview_and_the_status_of_Linux.pdf
  14. Memory-Centric Computing - Ethz, accessed April 19, 2025, https://people.inf.ethz.ch/omutlu/pub/onur-IEDM-3-4-Monday-MemoryCentricComputing-InvitedTalk-9-December-2024.pdf
  15. Databases in the Era of Memory-Centric Computing - VLDB Endowment, accessed April 19, 2025, https://www.vldb.org/cidrdb/papers/2025/p6-chronis.pdf
  16. Memory-centric Computing Systems: What's Old Is New Again - SIGARCH, accessed April 19, 2025, https://www.sigarch.org/memory-centric-computing-systems-whats-old-is-new-again/
  17. Next-Gen Interconnection Systems with Compute Express Link: a Comprehensive Survey, accessed April 19, 2025, https://arxiv.org/html/2412.20249v1
  18. How Flexible is CXL's Memory Protection? - ACM Queue, accessed April 19, 2025, https://queue.acm.org/detail.cfm?id=3606014
  19. How Flexible is CXL's Memory Protection? - University of Cambridge, accessed April 19, 2025, https://www.repository.cam.ac.uk/bitstreams/c56e69c4-e7d8-47a8-9cb3-769345eb0f8a/download
  20. CXL 3.0 - Everything You Need To Know [2023] - Logic Fruit Technologies, accessed April 19, 2025, https://www.logic-fruit.com/blog/cxl/cxl-3-0/
  21. CXL 1.0, 1.1. 2.0 3.0 - Compute Express Link - Serverparts.pl, accessed April 19, 2025, https://www.serverparts.pl/en/blog/cxl-10-11-20-30-compute-express-link-1
  22. Compute Express Link (CXL): All you need to know - Rambus, accessed April 19, 2025, https://www.rambus.com/blogs/compute-express-link/
  23. About CXL® - Compute Express Link, accessed April 19, 2025, https://computeexpresslink.org/about-cxl/
  24. Implementing CXL Memory on Linux on ThinkSystem V4 Servers - Lenovo Press, accessed April 19, 2025, https://lenovopress.lenovo.com/lp2184-implementing-cxl-memory-on-linux-on-thinksystem-v4-servers
  25. Compute Express Link - Wikipedia, accessed April 19, 2025, https://en.wikipedia.org/wiki/Compute_Express_Link
  26. Exploring Performance and Cost Optimization with ASIC-Based CXL Memory - OpenReview, accessed April 19, 2025, https://openreview.net/pdf?id=cJOoD0jx6b
  27. TPP: Transparent Page Placement for CXL-Enabled Tiered-Memory - SymbioticLab, accessed April 19, 2025, https://symbioticlab.org/publications/files/tpp:asplos23/tpp-asplos23.pdf
  28. Welcome to the Linux CXL documentation — CXL documentation, accessed April 19, 2025, https://linux-cxl.readthedocs.io/
  29. An Introduction to Compute Express Link (CXL) - MemVerge, accessed April 19, 2025, https://memverge.com/wp-content/uploads/2022/10/CXL-Forum-Wall-Street_MemVerge.pdf
  30. CXL Thriving As Memory Link - Semiconductor Engineering, accessed April 19, 2025, https://semiengineering.com/cxl-thriving-as-memory-link/
  31. Verifying CXL 3.1 Designs with Synopsys Verification IP, accessed April 19, 2025, https://www.synopsys.com/blogs/chip-design/verifying-cxl3-1-designs-with-synopsys-verification-ip.html
  32. Memory Sharing with CXL: Hardware and Software Design Approaches, accessed April 19, 2025, https://hcds-workshop.github.io/edition/2024/resources/Memory-Sharing-Jain-2024.pdf
  33. Memory Sharing with CXL: Hardware and Software Design Approaches - arXiv, accessed April 19, 2025, https://arxiv.org/html/2404.03245v1
  34. Memory Sharing with CXL: Hardware and Software Design Approaches - arXiv, accessed April 19, 2025, https://arxiv.org/pdf/2404.03245
  35. How Flexible Is CXL's Memory Protection? - Communications of the ACM, accessed April 19, 2025, https://cacm.acm.org/practice/how-flexible-is-cxls-memory-protection/
  36. CXL (Compute Express Link) Technology - Scientific Research Publishing, accessed April 19, 2025, https://www.scirp.org/journal/paperinformation?paperid=126038
  37. What is Compute Express Link (CXL)? - Trenton Systems, accessed April 19, 2025, https://www.trentonsystems.com/en-us/resource-hub/blog/what-is-compute-express-link-cxl
  38. Fabric Technology Required for Composable Memory - IntelliProp, accessed April 19, 2025, https://www.intelliprop.com/wp-content/uploads/2022/11/Composable-Memory-requires-a-Fabric-White-Paper.pdf
  39. Exploring and Evaluating Real-world CXL: Use Cases and System Adoption - arXiv, accessed April 19, 2025, https://arxiv.org/html/2405.14209v3
  40. Implementing CXL Memory on Linux on ThinkSystem V4 Servers - Lenovo Press, accessed April 19, 2025, https://lenovopress.lenovo.com/lp2184.pdf
  41. Octopus: Scalable Low-Cost CXL Memory Pooling | Request PDF - ResearchGate, accessed April 19, 2025, https://www.researchgate.net/publication/388067880_Octopus_Scalable_Low-Cost_CXL_Memory_Pooling
  42. Designing for the Future of System Architecture With CXL and Intel in the ATC - WWT, accessed April 19, 2025, https://www.wwt.com/article/designing-for-the-future-of-system-architecture-with-cxl-and-intel-in-the-atc
  43. CXL: The Future Of Memory Interconnect? - Semiconductor Engineering, accessed April 19, 2025, https://semiengineering.com/cxl-the-future-of-memory-interconnect/
  44. [2411.02282] A Comprehensive Simulation Framework for CXL Disaggregated Memory - arXiv, accessed April 19, 2025, https://arxiv.org/abs/2411.02282
  45. Compute Express Link (CXL) - Ayar Labs, accessed April 19, 2025, https://ayarlabs.com/glossary/compute-express-link-cxl/
  46. Architectural and System Implications of CXL-enabled Tiered Memory - arXiv, accessed April 19, 2025, https://arxiv.org/html/2503.17864v1
  47. CXL 2.0 and 3.0 for Storage and Memory Applications | Synopsys, accessed April 19, 2025, https://www.synopsys.com/designware-ip/technical-bulletin/cxl2-3-storage-memory-applications.html
  48. A CXL-Powered Database System: Opportunities and Challenges, accessed April 19, 2025, https://dbgroup.cs.tsinghua.edu.cn/ligl//papers/CXL_ICDE.pdf
  49. Explaining CXL Memory Pooling and Sharing - Compute Express Link, accessed April 19, 2025, https://computeexpresslink.org/blog/explaining-cxl-memory-pooling-and-sharing-1049/
  50. CXL 3.0: Revolutionizing Data Centre Memory - Optimize Performance & Reduce Costs, accessed April 19, 2025, https://www.ruijienetworks.com/support/tech-gallery/cxl3-0-solving-new-memory-problems-in-data-centres-part2
  51. An Open Industry Standard for Composable Computing - Compute Express LinkTM (CXL™), accessed April 19, 2025, https://computeexpresslink.org/wp-content/uploads/2023/12/CXL_FMS-2023-Tutorial_FINAL.pdf
  52. NVM Express® Support for CXL, accessed April 19, 2025, https://nvmexpress.org/wp-content/uploads/02_Martin-and-Molgaard_NVMe-Support-for-CXL_Final.pdf
  53. CXL Consortium Releases Compute Express Link 3.0 Specification to Expand Fabric Capabilities and Management - Business Wire, accessed April 19, 2025, https://www.businesswire.com/news/home/20220802005028/en/CXL-Consortium-Releases-Compute-Express-Link-3.0-Specification-to-Expand-Fabric-Capabilities-and-Management
  54. Compute Express Link (CXL) 3.0 Debuts, Wins CPU Interconnect Wars | Tom's Hardware, accessed April 19, 2025, https://www.tomshardware.com/news/cxl-30-debuts-one-cpu-interconnect-to-rule-them-all
  55. CXL 3.0 Specification Released - Doubles The Data Rate Of CXL 2.0 - Phoronix, accessed April 19, 2025, https://www.phoronix.com/news/CXL-3.0-Specification-Released
  56. CXL 3.0: Enabling composable systems with expanded fabric capabilities - YouTube, accessed April 19, 2025, https://www.youtube.com/watch?v=CIjDpazbtUU
  57. Pond: CXL-Based Memory Pooling Systems for Cloud Platforms - Microsoft, accessed April 19, 2025, https://www.microsoft.com/en-us/research/wp-content/uploads/2022/10/Pond-ASPLOS23.pdf
  58. Memory Disaggregation: Open Challenges in the Era of CXL - SymbioticLab, accessed April 19, 2025, https://symbioticlab.org/publications/files/disaggregation-future:hotinfra23/memory-disaggregation-hotinfra23.pdf
  59. Beyond processor-centric operating systems | USENIX, accessed April 19, 2025, https://www.usenix.org/system/files/conference/hotos15/hotos15-paper-faraboschi.pdf
  60. CXL 3.0 and Beyond: Advancements in Memory Management and Connectivity, accessed April 19, 2025, https://www.h3platform.com/blog-detail/58
  61. An Examination of CXL Memory Use Cases for In-Memory Database Management Systems using SAP HANA - VLDB Endowment, accessed April 19, 2025, https://www.vldb.org/pvldb/vol17/p3827-ahn.pdf
  62. Compute Express Link™ (CXL ™) Device Ecosystem and Usage Models, accessed April 19, 2025, https://computeexpresslink.org/wp-content/uploads/2023/12/CXL_FMS-Panel-2023_FINAL.pdf
  63. Panmnesia intros CXL 3.0-enabled memory sharing AI accelerator - Blocks and Files, accessed April 19, 2025, https://blocksandfiles.com/2023/11/27/panmnesia-has-cxl-3-0-enabled-memory-sharing-ai-accelerator/
  64. CXL Fabric Management - Compute Express Link, accessed April 19, 2025, https://computeexpresslink.org/blog/cxl-fabric-management-1089/
  65. CXL 2.0 / PCIe Gen 5 - The Future of Composable Infrastructure - H3 Platform, accessed April 19, 2025, https://www.h3platform.com/blog-detail/29
  66. CXL - Blocks and Files, accessed April 19, 2025, https://blocksandfiles.com/2022/04/20/cxl/
  67. CXL Glossary - Rambus, accessed April 19, 2025, https://www.rambus.com/interface-ip/cxl-glossary/
  68. Integrity and Data Encryption (IDE) Trends and Verification Challenges in CXL® (Compute Express Link®), accessed April 19, 2025, https://computeexpresslink.org/blog/integrity-and-data-encryption-ide-trends-and-verification-challenges-in-cxl-compute-express-link-2797/
  69. Compute Express Link (CXL) 3.0 Announced: Doubled Speeds and Flexible Fabrics, accessed April 19, 2025, https://www.anandtech.com/show/17520/compute-express-link-cxl-30-announced-doubled-speeds-and-flexible-fabrics
  70. CXL 3.0 Scales the Future Data Center - Verification - Cadence Blogs, accessed April 19, 2025, https://community.cadence.com/cadence_blogs_8/b/fv/posts/cxl-3-0-scales-the-future-data-center
  71. Compute Express Link™(CXL™) 3.0: Expanded capabilities for increasing scale and optimizing resource utilization - SNIA CMSS, accessed April 19, 2025, https://www.snia.org/sites/default/files/cmss/2023/SNIA-CMSS23-Rudoff-CXL-Expanded-Capabilities.pdf
  72. Introducing the CXL 3.1 Specification - Compute Express Link, accessed April 19, 2025, https://computeexpresslink.org/wp-content/uploads/2024/03/CXL_3.1-Webinar-Presentation_Feb_2024.pdf
  73. Enabling Efficient Transaction Processing on CXL-Based Memory Sharing - arXiv, accessed April 19, 2025, https://arxiv.org/html/2502.11046v1
  74. Compute Express Link (CXL)*: An open interconnect for HPC and AI applications - NOWLAB, accessed April 19, 2025, http://nowlab.cse.ohio-state.edu/static/media/workshops/presentations/exacomm24/Exacomm_ISC_2024_CXL_Debendra.pdf
  75. SDC2022 – Introducing CXL 3.0: Expanded Capabilities for Increased Scale and Optimized Resource Util - YouTube, accessed April 19, 2025, https://www.youtube.com/watch?v=X1sAyKo_28I
  76. CXL Standard Evolution: From CXL 2.0 to 3.1 | Synopsys Blog, accessed April 19, 2025, https://www.synopsys.com/blogs/chip-design/cxl-3-1-standard.html
  77. Hundreds of servers could share external memory pools across Panmnesia CXL fabric, accessed April 19, 2025, https://blocksandfiles.com/2024/08/01/panmnesia-cxl-fabric/
  78. Linux 6.14 CXL Updates Make Preparations Around Type 2 Support & CXL 3.1 - Phoronix, accessed April 19, 2025, https://www.phoronix.com/news/Linux-6.14-CXL
  79. UnifabriX taking CXL external memory mainstream - Blocks and Files, accessed April 19, 2025, https://blocksandfiles.com/2025/01/15/unifabrix-taking-cxl-external-memory-mainstream/
  80. CXL Update Emphasizes Security [Byline] - Gary Hilson, accessed April 19, 2025, https://hilson.ca/cxl-update-emphasizes-security-byline/
  81. CXL Update Emphasizes Security - EE Times, accessed April 19, 2025, https://www.eetimes.com/cxl-update-emphasizes-security/
  82. Unlocking CXL's Potential: Revolutionizing Server Memory and Performance - SNIA.org, accessed April 19, 2025, https://snia.org/sites/default/files/CMSC/2025-0326_Unlocking_CXL_Webinar_Final.pdf
  83. What are the new features in the CXL 3.0 specification? - ASTERA LABS, INC., accessed April 19, 2025, https://www.asteralabs.com/faqs/what-are-the-new-features-in-cxl-3-0-specification/
  84. A Case Against CXL Memory Pooling - Events, accessed April 19, 2025, https://conferences.sigcomm.org/hotnets/2023/papers/hotnets23_levis.pdf
  85. Octopus: Scalable Low-Cost CXL Memory Pooling - arXiv, accessed April 19, 2025, https://arxiv.org/pdf/2501.09020
  86. CXL GFAM Global Fabric Attached Memory Device - ServeTheHome, accessed April 19, 2025, https://www.servethehome.com/compute-express-link-cxl-3-0-is-the-exciting-building-block-for-disaggregation/cxl-gfam-global-fabric-attached-memory-device/
  87. Logical Memory Pools: Flexible and Local Disaggregated Memory, accessed April 19, 2025, https://conferences.sigcomm.org/hotnets/2023/papers/hotnets23_amaro.pdf
  88. How CXL and Memory Pooling Reduce HPC Latency | Synopsys Blog, accessed April 19, 2025, https://www.synopsys.com/blogs/chip-design/cxl-protocol-memory-pooling.html
  89. A Comprehensive Simulation Framework for CXL Disaggregated Memory - arXiv, accessed April 19, 2025, https://arxiv.org/html/2411.02282v2
  90. Exploring and Evaluating Real-world CXL: Use Cases and System Adoption - arXiv, accessed April 19, 2025, https://arxiv.org/html/2405.14209v1
  91. Glossary — Intel Unified Memory Framework 0.12.0 documentation - GitHub Pages, accessed April 19, 2025, https://oneapi-src.github.io/unified-memory-framework/glossary.html
  92. A Comprehensive Simulation Framework for CXL Disaggregated Memory - arXiv, accessed April 19, 2025, https://arxiv.org/html/2411.02282v5
  93. Arm CMN S3: Driving CXL Storage Innovation - Servers and Cloud Computing blog, accessed April 19, 2025, https://community.arm.com/arm-community-blogs/b/servers-and-cloud-computing-blog/posts/arm-cmn-s3-driving-cxl-storage-innovation
  94. Compute Express Link - Neoverse Reference Design Platform Software - Arm, accessed April 19, 2025, https://neoverse-reference-design.docs.arm.com/en/latest/features/cxl.html
  95. CXL Security (Training) - MindShare, accessed April 19, 2025, https://www.mindshare.com/Learn/CXL_Security
  96. NeoMem: Hardware/Software Co-Design for CXL-Native Memory Tiering - arXiv, accessed April 19, 2025, https://arxiv.org/html/2403.18702v2
  97. Formalising CXL Cache Coherence - Imperial College London, accessed April 19, 2025, https://www.doc.ic.ac.uk/~afd/papers/2025/ASPLOS-CXL.pdf
  98. Compute Express Link Subsystem Maturity Map - The Linux Kernel documentation, accessed April 19, 2025, https://docs.kernel.org/driver-api/cxl/maturity-map.html
  99. Pond: CXL-Based Memory Pooling Systems for Cloud Platforms - Microsoft, accessed April 19, 2025, https://www.microsoft.com/en-us/research/wp-content/uploads/2022/10/2023_Pond_asplos23_official_asplos_version.pdf
  100. Dissecting CXL Memory Performance at Scale: Analysis, Modeling, and Optimization - arXiv, accessed April 19, 2025, https://arxiv.org/html/2409.14317v1
  101. Demystifying CXL Memory with Genuine CXL-Ready Systems and Devices - arXiv, accessed April 19, 2025, https://arxiv.org/pdf/2303.15375
  102. CXL Memory Pooling will Save Millions in DRAM Cost | TechPowerUp Forums, accessed April 19, 2025, https://www.techpowerup.com/forums/threads/cxl-memory-pooling-will-save-millions-in-dram-cost.296786/
  103. Architectural and System Implications of CXL-enabled Tiered Memory - arXiv, accessed April 19, 2025, https://arxiv.org/html/2503.17864v2
  104. Systematic CXL Memory Characterization and Performance Analysis at Scale - People, accessed April 19, 2025, https://people.cs.vt.edu/jinshu/docs/papers/Melody_ASPLOS.pdf
  105. Managing Memory Tiers with CXL in Virtualized Environments - USENIX, accessed April 19, 2025, https://www.usenix.org/system/files/osdi24-zhong-yuhong.pdf
  106. FPGA-based Emulation and Device-Side Management for CXL-based Memory Tiering Systems - arXiv, accessed April 19, 2025, https://arxiv.org/html/2502.19233v2
  107. Architectural and System Implications of CXL-enabled Tiered Memory - arXiv, accessed April 19, 2025, https://arxiv.org/pdf/2503.17864
  108. Pond: CXL-Based Memory Pooling Systems for Cloud Platforms (ASPLOS'23) - GitHub, accessed April 19, 2025, https://github.com/MoatLab/Pond
  109. A Comprehensive Simulation Framework for CXL Disaggregated Memory | Request PDF, accessed April 19, 2025, https://www.researchgate.net/publication/385560282_A_Comprehensive_Simulation_Framework_for_CXL_Disaggregated_Memory
  110. www.eetimes.com, accessed April 19, 2025, [https://www.eetimes.com/cxl-update-emphasizes-security/#:~:text=The%20trusted%20security%20protocol%20(TSP,to%20host%20confidential%20computing%20workloads.](https://www.eetimes.com/cxl-update-emphasizes-security/#:~:text=The trusted security protocol (TSP,to host confidential computing workloads.)
  111. Dissecting CXL Memory Performance at Scale: Analysis, Modeling, and Optimization - arXiv, accessed April 19, 2025, https://arxiv.org/pdf/2409.14317
  112. Compute Express Link (CXL) — QEMU documentation, accessed April 19, 2025, https://www.qemu.org/docs/master/system/devices/cxl.html
  113. A Practical Guide to Identify Compute Express Link (CXL) Devices in Your Server, accessed April 19, 2025, https://stevescargall.com/blog/2023/05/a-practical-guide-to-identify-compute-express-link-cxl-devices-in-your-server/
  114. Toward CXL-Native Memory Tiering via Device-Side Profiling - arXiv, accessed April 19, 2025, https://arxiv.org/html/2403.18702v1
  115. Beware, PCIe Switches! CXL Pools Are Out to Get You - arXiv, accessed April 19, 2025, https://arxiv.org/html/2503.23611v1
  116. 9.2. Automatic NUMA Balancing | Red Hat Product Documentation, accessed April 19, 2025, https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/7/html/virtualization_tuning_and_optimization_guide/sect-virtualization_tuning_optimization_guide-numa-auto_numa_balancing
  117. TPP: Transparent Page Placement for CXL-Enabled Tiered-Memory - Meta Research, accessed April 19, 2025, https://research.facebook.com/publications/tpp-transparent-page-placement-for-cxl-enabled-tiered-memory/
  118. [2206.02878] TPP: Transparent Page Placement for CXL-Enabled Tiered-Memory - arXiv, accessed April 19, 2025, https://arxiv.org/abs/2206.02878
  119. Nomad: Non-Exclusive Memory Tiering via Transactional Page Migration - USENIX, accessed April 19, 2025, https://www.usenix.org/system/files/osdi24-xiang.pdf
  120. Lightweight Frequency-Based Tiering for CXL Memory Systems - arXiv, accessed April 19, 2025, https://arxiv.org/html/2312.04789v1
  121. CXL Memory Disaggregation and Tiering: Lessons Learned from Storage - SNIA.org, accessed April 19, 2025, https://www.snia.org/educational-library/cxl-memory-disaggregation-and-tiering-lessons-learned-storage-2023
  122. NeoMem: Hardware/Software Co-Design for CXL-Native Memory Tiering - Microsoft, accessed April 19, 2025, https://www.microsoft.com/en-us/research/publication/neomem-hardware-software-co-design-for-cxl-native-memory-tiering/
  123. Managing Memory Tiers with CXL in Virtualized Environments - USENIX, accessed April 19, 2025, https://www.usenix.org/conference/osdi24/presentation/zhong-yuhong
  124. Tiresias: Optimizing NUMA Performance with CXL Memory and Locality-Aware Process Scheduling - Temple CIS, accessed April 19, 2025, https://cis.temple.edu/~wu/research/publications/Publication_files/acmturc24-12.pdf
  125. Using Linux Kernel Tiering with Compute Express Link (CXL) Memory - Steve Scargall, accessed April 19, 2025, https://stevescargall.com/blog/2024/05/using-linux-kernel-tiering-with-compute-express-link-cxl-memory/
  126. Re: [PATCH -V2] cxl/region: Support to calculate memory tier abstract distance - The Linux-Kernel Archive, accessed April 19, 2025, https://lkml.iu.edu/2406.1/04010.html
  127. MemVerge: Homepage, accessed April 19, 2025, https://memverge.com/
  128. Memory Machine – CXL - MemVerge, accessed April 19, 2025, https://memverge.com/memory-machine-cxl/
  129. Breaking Memory Barriers | Compute Express Link, accessed April 19, 2025, https://computeexpresslink.org/wp-content/uploads/2024/12/CXL-Breaking-Memory-Barriers-Webinar.pdf
  130. Re: [PATCH v5 0/8] DAMON based tiered memory management for CXL memory - The Linux-Kernel Archive, accessed April 19, 2025, https://lkml.iu.edu/2406.1/07403.html
  131. [PDF] Page Table Management for Heterogeneous Memory Systems - Semantic Scholar, accessed April 19, 2025, https://www.semanticscholar.org/paper/7f81a1c543e07f892fe10d00e1781eace1592f67
  132. [2103.10779] Page Table Management for Heterogeneous Memory Systems - arXiv, accessed April 19, 2025, https://arxiv.org/abs/2103.10779
  133. Unleashing the Future of Memory Management: Exploring CXL Dynamic Capacity Devices with Docker and QEMU - MemVerge, accessed April 19, 2025, https://memverge.com/unleashing-the-future-of-memory-management/
  134. Jackrabbit Labs - the Future of Memory and Storage, accessed April 19, 2025, https://files.futurememorystorage.com/proceedings/2024/20240806_CXLT-102-1_Mackey.pdf
  135. OCP CMS Logical System Architecture White Paper - Open Compute Project, accessed April 19, 2025, https://www.opencompute.org/documents/ocp-cms-logical-system-architecture-white-paper-pdf-1
  136. Introducing Omega Fabric Based on CXL - IntelliProp, accessed April 19, 2025, https://www.intelliprop.com/products-page
  137. Compute Express Link Memory Devices - The Linux Kernel documentation, accessed April 19, 2025, https://docs.kernel.org/driver-api/cxl/memory-devices.html
  138. LPC2022: Meta's CXL Journey and Learnings - Linux Plumbers Conference, accessed April 19, 2025, [https://lpc.events/event/16/contributions/1207/attachments/950/1866/LPC2022_%20Meta's%20CXL%20Journey%20and%20Learnings.pdf](https://lpc.events/event/16/contributions/1207/attachments/950/1866/LPC2022_ Meta's CXL Journey and Learnings.pdf)
  139. Compute Express Link Memory Devices - The Linux Kernel Archives, accessed April 19, 2025, https://www.kernel.org/doc/html/v6.1/driver-api/cxl/memory-devices.html
  140. CXL Fabric Management Standards | PPT - SlideShare, accessed April 19, 2025, https://www.slideshare.net/slideshow/cxl-fabric-management-standards/262688178
  141. libcxlmi a CXL Management Interace library - Linux Plumbers Conference, accessed April 19, 2025, https://lpc.events/event/18/contributions/1876/attachments/1441/3072/lpc24-dbueso-libcxlmi.pdf
  142. computexpresslink/libcxlmi: CXL Management Interface library - GitHub, accessed April 19, 2025, https://github.com/computexpresslink/libcxlmi
  143. EDM: An Ultra-Low Latency Ethernet Fabric for Memory Disaggregation - arXiv, accessed April 19, 2025, https://arxiv.org/html/2411.08300v4
  144. ¯Apta: Fault-tolerant object-granular CXL disaggregated memory for accelerating FaaS - cs.utah.edu, accessed April 19, 2025, https://users.cs.utah.edu/~vijay/papers/dsn23.pdf
  145. EDM: An Ultra-Low Latency Ethernet Fabric for Memory Disaggregation - arXiv, accessed April 19, 2025, https://arxiv.org/html/2411.08300v1
  146. arXiv:2203.00241v4 [cs.OS] 21 Oct 2022, accessed April 19, 2025, https://arxiv.org/pdf/2203.00241
  147. Survey of Disaggregated Memory: Cross-layer Technique Insights for Next-Generation Datacenters - arXiv, accessed April 19, 2025, https://arxiv.org/html/2503.20275v1
  148. CXL-ANNS: Software-Hardware Collaborative Memory Disaggregation and Computation for Billion-Scale Approximate Nearest Neighbor Search - USENIX, accessed April 19, 2025, https://www.usenix.org/system/files/atc23-jang.pdf
  149. SkyByte: Architecting an Efficient Memory-Semantic CXL-based SSD with OS and Hardware Co-design - arXiv, accessed April 19, 2025, https://arxiv.org/html/2501.10682v1
  150. CXL Enumeration: How Are Devices Discovered in System Fabric? - Cadence Blogs, accessed April 19, 2025, https://community.cadence.com/cadence_blogs_8/b/fv/posts/cxl-enumeration-how-are-devices-discovered-in-system-fabric
  151. The Fascinating Path of CXL 2.0 Device Discovery | Synopsys Blog, accessed April 19, 2025, https://www.synopsys.com/blogs/chip-design/cxl-2-device-discovery-path.html
  152. Compute Express Link Memory Devices - The Linux Kernel documentation, accessed April 19, 2025, https://docs.kernel.org/6.3/driver-api/cxl/memory-devices.html
  153. cxl-reskit/cxl-reskit: CXL Memory Resource Kit top-level repository - GitHub, accessed April 19, 2025, https://github.com/cxl-reskit/cxl-reskit
  154. cxl(1) - NDCTL User Guide, accessed April 19, 2025, https://docs.pmem.io/ndctl-user-guide/v72.1/cxl-man-pages/cxl-1
  155. CXL Address Translation Support For AMD Zen 5 Sees Linux Patches - Phoronix, accessed April 19, 2025, https://www.phoronix.com/news/AMD-Zen5-CXL-Translation-v1
  156. Compute Express LinkTM (CXLTM), accessed April 19, 2025, https://computeexpresslink.org/wp-content/uploads/2024/02/20210812_Type3_MGMT_using_MCTP_CCI_ECN-errata-update20211116.pdf)))

GTC Beyond CUDA

1. Introduction

1.1 Setting the Stage: NVIDIA's CUDA and its Dominance in AI Compute

NVIDIA Corporation, initially renowned for its graphics processing units (GPUs) powering the gaming industry, strategically pivoted over the last two decades to become the dominant force in artificial intelligence (AI) computing. A cornerstone of this transformation was the introduction of the Compute Unified Device Architecture (CUDA) in 2006. CUDA is far more than just a programming language; it represents NVIDIA's proprietary parallel computing platform and a comprehensive software ecosystem, encompassing compilers, debuggers, profilers, extensive libraries (like cuDNN for deep learning and cuBLAS for linear algebra), and development tools. This ecosystem unlocked the potential of GPUs for general-purpose processing (GPGPU), enabling developers to harness the massive parallelism inherent in NVIDIA hardware for computationally intensive tasks far beyond graphics rendering.

This strategic focus on software and hardware synergy has propelled NVIDIA to a commanding position in the AI market. Estimates consistently place NVIDIA's share of the AI accelerator and data center GPU market between 70% and 95%, with recent figures often citing 80% to 92% dominance. This market leadership is reflected in staggering financial growth, with data center revenue surging, exemplified by figures like $18.4 billion in a single quarter of 2023. High-performance GPUs like the A100, H100, and the upcoming Blackwell series have become the workhorses for training and deploying large-scale AI models, utilized by virtually all major technology companies and research institutions, including OpenAI, Google, and Meta. Consequently, CUDA has solidified its status as the de facto standard programming environment for GPU-accelerated computing, particularly within the AI domain, underpinning widely used frameworks like PyTorch and TensorFlow.

1.2 The Emerging "Beyond CUDA" Narrative: GTC Insights and Industry Momentum

Despite NVIDIA's entrenched position, a narrative exploring computational pathways "Beyond CUDA" is gaining traction, even surfacing within NVIDIA's own GPU Technology Conference (GTC) events. The focus of the provided GTC video segment, starting from the 5 minute 27 second mark, on alternatives signifies that the discussion around diversifying the AI compute stack is relevant and acknowledged within the broader ecosystem [User Query].

This internal discussion is mirrored and amplified by external industry movements. Notably, the "Beyond CUDA Summit," organized by TensorWave (a cloud provider utilizing AMD accelerators) and featuring prominent figures like computer architects Jim Keller and Raja Koduri, explicitly aimed to challenge NVIDIA's dominance. This event, strategically held near NVIDIA's GTC 2025, centered on dissecting the "CUDA moat" and exploring viable alternatives, underscoring a growing industry-wide desire for greater hardware flexibility, cost efficiency, and reduced vendor lock-in.

1.3 Report Objectives and Structure

This report aims to provide an expert-level analysis of the evolving AI compute landscape, moving beyond the CUDA-centric view. It will dissect the concept of the "CUDA moat," examine the strategies being employed to challenge NVIDIA's dominance, and detail the alternative hardware and software solutions emerging across the AI workflow – encompassing pre-training, post-training (optimization and fine-tuning), and inference.

The analysis will draw upon insights derived from the specified GTC video segment, synthesizing this information with data and perspectives gathered from recent industry reports, technical analyses, and market commentary found in the provided research materials. The report is structured into the following key sections:

  • Crossing the Moat: Deconstructing CUDA's competitive advantages and analyzing industry strategies for diversification.
  • Pre-training Beyond CUDA: Examining alternative hardware and software for large-scale model training.
  • Post-training Beyond CUDA: Investigating non-CUDA tools and techniques for model optimization and fine-tuning.
  • Inference Beyond CUDA: Detailing the diverse hardware and software solutions for deploying models outside the CUDA ecosystem.
  • Industry Outlook and Conclusion: Assessing the current market dynamics, adoption trends, and the future trajectory of AI compute heterogeneity.

2. Crossing the Moat: Understanding and Challenging CUDA's Dominance

2.1 Historical Context and the Rise of CUDA

NVIDIA's journey to AI dominance was significantly shaped by the strategic introduction of CUDA in 2006. This platform marked a pivotal shift, enabling developers to utilize the parallel processing power of NVIDIA GPUs for general-purpose computing tasks, extending their application far beyond traditional graphics rendering. NVIDIA recognized the potential of parallel computing on its hardware architecture early on, developing CUDA as a proprietary platform to unlock this capability. This foresight, driven partly by academic research demonstrating GPU potential for scientific computing and initiatives like the Brook streaming language developed by future CUDA creator Ian Buck , provided NVIDIA with a crucial first-mover advantage.

CUDA was designed with developers in mind, abstracting away much of the underlying hardware complexity and allowing researchers and engineers to focus more on algorithms and applications rather than intricate hardware nuances. It provided APIs, libraries, and tools within familiar programming paradigms (initially C/C++, later Fortran and Python). Over more than a decade, CUDA matured with relatively limited competition from viable, comprehensive alternatives. This extended period allowed the platform and its ecosystem to become deeply embedded in academic research, high-performance computing (HPC), and, most significantly, the burgeoning field of AI.

2.2 Deconstructing the "CUDA Moat": Ecosystem, Lock-in, and Performance

The term "CUDA moat" refers to the collection of sustainable competitive advantages that protect NVIDIA's dominant position in the AI compute market, primarily derived from its tightly integrated hardware and software ecosystem. This moat is multifaceted:

  • Ecosystem Breadth and Network Effects:

    The CUDA ecosystem is vast, encompassing millions of developers worldwide, thousands of companies, and a rich collection of optimized libraries (e.g., cuDNN, cuBLAS, TensorRT), sophisticated development and profiling tools, extensive documentation, and strong community support.

    CUDA is also heavily integrated into academic curricula, ensuring a steady stream of new talent proficient in NVIDIA's tools.

    This widespread adoption creates powerful network effects: as more developers and applications utilize CUDA, more tools and resources are created for it, further increasing its value and reinforcing its position as the standard.

  • High Switching Costs and Developer Inertia:

    Companies and research groups have invested heavily in developing, testing, and optimizing codebases built upon CUDA.

    Migrating these complex workflows to alternative platforms like AMD's ROCm or Intel's oneAPI represents a daunting task. It often requires significant code rewriting, retraining developers on new tools and languages, and introduces substantial risks related to achieving comparable performance, stability, and correctness.

    This "inherent inertia" within established software ecosystems creates high switching costs, making organizations deeply reluctant to abandon their CUDA investments, even if alternatives offer potential benefits.

  • Performance Optimization and Hardware Integration:

    CUDA provides developers with low-level access to NVIDIA GPU hardware, enabling fine-grained optimization to extract maximum performance.

    This is critical in compute-intensive AI workloads. The tight integration between CUDA software and NVIDIA hardware features, such as Tensor Cores (specialized units for matrix multiplication), allows for significant acceleration.

    Competitors often struggle to match this level of performance tuning due to the deep co-design of NVIDIA's hardware and software.

    While programming Tensor Cores directly can involve "arcane knowledge" and dealing with undocumented behaviors

    , the availability of libraries like cuBLAS and CUTLASS abstracts some of this complexity.

  • Backward Compatibility:

    NVIDIA has generally maintained backward compatibility for CUDA, allowing older code to run on newer GPU generations (though limitations exist, as newer CUDA versions require specific drivers and drop support for legacy hardware over time).

    This perceived stability encourages long-term investment in the CUDA platform.

  • Vendor Lock-in:

    The cumulative effect of this deep ecosystem, high switching costs, performance advantages on NVIDIA hardware, and established workflows results in significant vendor lock-in.

    Developers and organizations become dependent on NVIDIA's proprietary platform, limiting hardware choices, potentially stifling competition, and giving NVIDIA considerable market power.

2.3 Industry Strategies for Diversification

Recognizing the challenges posed by the CUDA moat, various industry players are pursuing strategies to foster a more diverse and open AI compute ecosystem. These efforts span competitor platform development, the promotion of open standards and abstraction layers, and initiatives by large-scale users.

  • Competitor Platform Development:

    • AMD ROCm (Radeon Open Compute):

      AMD's primary answer to CUDA is ROCm, an open-source software stack for GPU computing.

      Key to its strategy is the Heterogeneous-computing Interface for Portability (HIP), designed to be syntactically similar to CUDA, easing code migration.

      AMD provides the HIPIFY tool to automate the conversion of CUDA source code to HIP C++, although manual adjustments are often necessary.

      Despite progress, ROCm has faced significant challenges. Historically, it supported a limited range of AMD GPUs, suffered from stability issues and performance gaps compared to CUDA, and lagged in adopting new features and supporting the latest hardware.

      However, AMD continues to invest heavily in ROCm, improving framework support (e.g., native PyTorch integration

      ), expanding hardware compatibility (including consumer GPUs, albeit sometimes unofficially or with delays

      ), and achieving notable adoption for its Instinct MI300 series accelerators by major hyperscalers.

    • Intel oneAPI:

      Intel promotes oneAPI as an open, unified, cross-architecture programming model based on industry standards, particularly SYCL (Data Parallel C++ or DPC++).

      It aims to provide portability across diverse hardware types, including CPUs, GPUs (Intel integrated and discrete), FPGAs, and other accelerators, explicitly positioning itself as an alternative to CUDA lock-in.

      oneAPI is backed by the Unified Acceleration (UXL) Foundation, involving multiple companies.

      While offering a promising vision for heterogeneity, oneAPI is a relatively newer initiative compared to CUDA and faces the challenge of building a comparable ecosystem and achieving widespread adoption.

    • Other Initiatives:

      OpenCL, an earlier open standard for heterogeneous computing, remains relevant, particularly in mobile and embedded systems, but has struggled to gain traction in high-performance AI due to fragmentation, slow evolution, and performance limitations compared to CUDA.

      Vulkan Compute, leveraging the Vulkan graphics API, offers low-level control and potential performance benefits but has a steeper learning curve and a less mature ecosystem for general-purpose compute.

      Newer entrants like Modular Inc.'s Mojo programming language and MAX platform aim to combine Python's usability with C/CUDA performance, targeting AI hardware programmability directly.

  • Open Standards and Abstraction Layers:

    • A significant trend involves leveraging higher-level AI frameworks like PyTorch, TensorFlow, and JAX, which can potentially abstract away underlying hardware specifics.

      If a model is written in PyTorch, the ideal scenario is that it can run efficiently on NVIDIA, AMD, or Intel hardware simply by targeting the appropriate backend (CUDA, ROCm, oneAPI/SYCL).

    • The development of PyTorch 2.0, featuring TorchDynamo for graph capture and TorchInductor as a compiler backend, represents a move towards greater flexibility.

      TorchInductor can generate code for different backends, including using OpenAI Triton for GPUs or OpenMP/C++ for CPUs, potentially reducing direct dependence on CUDA libraries for certain operations.

    • OpenAI Triton itself is positioned as a Python-like language and compiler for writing high-performance custom GPU kernels, aiming to achieve performance comparable to CUDA C++ but with significantly improved developer productivity.

      While currently focused on NVIDIA GPUs, its open-source nature holds potential for broader hardware support.

    • OpenXLA (Accelerated Linear Algebra), originating from Google's XLA compiler used in TensorFlow and JAX, is another initiative focused on creating a compiler ecosystem that can target diverse hardware backends.

    • However, these abstraction layers are not a panacea. The abstraction is often imperfect ("leaky"), many essential libraries within the framework ecosystems are still optimized primarily for CUDA or lack robust support for alternatives, performance parity is not guaranteed, and NVIDIA exerts considerable influence on the development roadmap of frameworks like PyTorch, potentially steering them in ways that favor CUDA.

      Achieving true first-class support for alternative backends within these dominant frameworks remains a critical challenge.

  • Hyperscaler Initiatives: The largest consumers of AI hardware – cloud hyperscalers like Google (TPUs), AWS (Trainium, Inferentia), Meta, and Microsoft – have the resources and motivation to develop their own custom AI silicon and potentially accompanying software stacks. This strategy allows them to optimize hardware for their specific workloads, control their supply chain, reduce costs, and crucially, avoid long-term dependence on NVIDIA. Their decisions to adopt competitor hardware (like AMD MI300X ) or build in-house solutions represent perhaps the most significant direct threat to the CUDA moat's long-term durability.

  • Direct Low-Level Programming (PTX): For organizations seeking maximum performance and control, bypassing CUDA entirely and programming directly in NVIDIA's assembly-like Parallel Thread Execution (PTX) language is an option, as demonstrated by DeepSeek AI. PTX acts as an intermediate representation between high-level CUDA code and the GPU's machine code. While this allows for fine-grained optimization potentially exceeding standard CUDA libraries, PTX is only partially documented, changes between GPU generations, and is even more tightly locked to NVIDIA hardware, making it a highly complex and specialized approach unsuitable for most developers.

2.4 Implications of the Competitive Landscape

The analysis of CUDA's dominance and the strategies to counter it reveals several key points about the competitive dynamics. Firstly, the resilience of NVIDIA's market position stems less from insurmountable technical superiority in every aspect and more from the profound inertia within the software ecosystem. The vast investment in CUDA codebases, developer skills, and tooling creates significant friction against adopting alternatives. This suggests that successful competitors need not only technically competent hardware but also a superior developer experience, seamless migration paths, robust framework integration, and compelling value propositions (e.g., cost, specific features) to overcome this inertia.

Secondly, abstraction layers like PyTorch and compilers like Triton present a complex scenario. While they hold the promise of hardware agnosticism, potentially weakening the direct CUDA lock-in, NVIDIA's deep integration and influence within these ecosystems mean they can also inadvertently reinforce the moat. The best-supported, highest-performing path often remains via CUDA. The ultimate impact of these layers depends critically on whether alternative hardware vendors can achieve true first-class citizenship and performance parity within them.

Thirdly, the "Beyond CUDA" movement suffers from fragmentation. The existence of multiple competing alternatives (ROCm, oneAPI, OpenCL, Vulkan Compute, Mojo, etc.) risks diluting development efforts and hindering the ability of any single alternative to achieve the critical mass needed to effectively challenge the unified CUDA front. This mirrors the historical challenges faced by OpenCL due to vendor fragmentation and lack of unified direction. Overcoming this may require market consolidation or the emergence of clear winners for specific niches.

Finally, the hyperscale cloud providers represent a powerful disruptive force. Their immense scale, financial resources, and strategic imperative to avoid vendor lock-in position them uniquely to alter the market dynamics. Their adoption of alternative hardware or the development of proprietary silicon and software stacks could create viable alternative ecosystems much faster than traditional hardware competitors acting alone.

Table 2.1: CUDA Moat Components and Counter-Strategies

Moat Component NVIDIA's Advantage Competitor Strategies Key Challenges for Competitors
Ecosystem Size Millions of developers, vast community, academic integration Build communities around ROCm/oneAPI/Mojo; Leverage open-source framework communities (PyTorch, TF) Reaching critical mass; Overcoming established network effects; Competing with NVIDIA's resources
Library Maturity Highly optimized, extensive libraries (cuDNN, cuBLAS, TensorRT) Develop competing libraries (ROCm libraries, oneAPI libraries); Contribute to framework-level ops Achieving feature/performance parity; Ensuring stability and robustness; Breadth of domain coverage
Developer Familiarity Decades of use, established workflows, available talent pool Simplify APIs (e.g., HIP similarity to CUDA); Provide migration tools (HIPIFY, SYCLomatic); Focus on usability Overcoming learning curves; Convincing developers of stability/benefits; Retraining workforce
Performance Optimization Tight hardware-software co-design; Low-level access; Tensor Core integration Optimize ROCm/oneAPI compilers; Improve framework backend performance; Develop specialized hardware Matching NVIDIA's optimization level; Accessing/optimizing specialized hardware features (like Tensor Cores)
Switching Costs High cost/risk of rewriting code, retraining, validating Provide automated porting tools; Ensure framework compatibility; Offer significant cost/performance benefits Imperfect porting tools; Ensuring functional equivalence and performance; Justifying the migration effort
Framework Integration Deep integration & influence in PyTorch/TF; Optimized paths Achieve native, high-performance support in frameworks; Leverage open-source contributions Competing with NVIDIA's influence; Ensuring timely support for new framework features; Library dependencies
Hyperscaler Dependence Major cloud providers are largest customers, rely on CUDA Hyperscalers adopt AMD/Intel; Develop custom silicon/software; Promote open standards Hyperscalers' internal efforts may not benefit broader market; Competing for hyperscaler design wins

3. Pre-training Beyond CUDA

3.1 Challenges in Pre-training

The pre-training phase for state-of-the-art AI models, particularly large language models (LLMs) and foundation models, involves computations at an immense scale. This process demands not only massive parallel processing capabilities but also exceptional stability and reliability over extended periods, often weeks or months. Historically, the maturity, performance, and robustness of NVIDIA's hardware coupled with the CUDA ecosystem made it the overwhelmingly preferred choice for these demanding tasks, establishing a high bar for any potential alternatives.

3.2 Alternative Hardware Accelerators

Despite NVIDIA's dominance, several alternative hardware platforms are being positioned and increasingly adopted for large-scale AI pre-training:

  • AMD Instinct Series (MI200, MI300X/MI325):

    AMD's Instinct line, particularly the MI300 series, directly targets NVIDIA's high-end data center GPUs like the A100 and H100.

    These accelerators offer competitive specifications, particularly in areas like memory capacity and bandwidth, which are critical for large models. They have gained traction with major hyperscalers, including Microsoft Azure, Oracle Cloud, and Meta, who see them as a viable alternative to reduce reliance on NVIDIA and potentially lower costs.

    Cloud platforms like TensorWave are also building services based on AMD Instinct hardware.

    AMD emphasizes a strategy centered around open standards and cost-effectiveness compared to NVIDIA's offerings.

  • Intel Gaudi Accelerators (Gaudi 2, Gaudi 3):

    Intel's Gaudi family represents dedicated ASICs designed specifically for AI training and inference workloads.

    Intel markets Gaudi accelerators, such as the recent Gaudi 3, as a significantly more cost-effective alternative to NVIDIA's flagship GPUs, aiming to capture a segment of the market prioritizing value.

    Gaudi accelerators feature integrated high-speed networking (Ethernet), facilitating the construction of large training clusters.

    It's noteworthy that deploying models on Gaudi often relies on Intel's specific SynapseAI software stack, which may differ from the broader oneAPI initiative in some aspects.

  • Google TPUs (Tensor Processing Units):

    Developed in-house by Google, TPUs are custom ASICs highly optimized for TensorFlow and JAX workloads.

    They have been instrumental in training many of Google's largest models and are available through Google Cloud Platform. TPUs demonstrate the potential of domain-specific architectures tailored explicitly for machine learning computations.

  • Other Emerging Architectures:

    The landscape is further diversifying with other players. Amazon Web Services (AWS) offers its Trainium chips for training.

    Reports suggest OpenAI and Microsoft may be developing their own custom AI accelerators.

    Startups like Cerebras Systems (with wafer-scale engines) and Groq (focused on low-latency inference, but indicative of architectural innovation) are exploring novel designs.

    Huawei also competes with its Ascend AI chips, particularly in the Chinese market, based on its Da Vinci architecture.

    This proliferation of hardware underscores the intense interest and investment in finding alternatives or complements to NVIDIA's GPUs.

3.3 Software Stacks for Large-Scale Training

Hardware alone is insufficient; robust software stacks are essential to harness these accelerators for pre-training:

  • ROCm Ecosystem:

    Training on AMD Instinct GPUs primarily relies on the ROCm software stack, particularly its integration with major AI frameworks like PyTorch and TensorFlow.

    While functional and improving, the ROCm ecosystem's maturity, ease of use, breadth of library support, and performance consistency have historically been points of concern compared to the highly refined CUDA ecosystem.

    Success hinges on continued improvements in ROCm's stability and performance within these critical frameworks.

  • oneAPI and Supporting Libraries:

    Intel's oneAPI aims to provide the software foundation for training on its diverse hardware portfolio (CPUs, GPUs, Gaudi accelerators).

    It utilizes DPC++ (based on SYCL) as the core language and includes libraries optimized for deep learning tasks, integrating with frameworks like PyTorch and TensorFlow.

    The goal is a unified programming experience across different Intel architectures, simplifying development for heterogeneous environments.

  • Leveraging PyTorch/JAX/TensorFlow with Alternative Backends:

    Regardless of the underlying hardware (AMD, Intel, Google TPU), the primary interface for most researchers and developers conducting large-scale pre-training remains high-level frameworks like PyTorch, JAX, or TensorFlow.

    The viability of non-NVIDIA hardware for pre-training is therefore heavily dependent on the quality, performance, and completeness of the respective framework backends (e.g., PyTorch on ROCm, JAX on TPU, TensorFlow on oneAPI).

  • The Role of Compilers (Triton, XLA):

    Compilers play a crucial role in bridging the gap between high-level framework code and low-level hardware execution. OpenAI Triton, used as a backend component within PyTorch 2.0's Inductor, translates Python-based operations into efficient GPU code (currently PTX for NVIDIA, but potentially adaptable).

    Similarly, XLA optimizes and compiles TensorFlow and JAX graphs for various targets, including TPUs and GPUs.

    The efficiency and target-awareness of these compilers are critical for achieving high performance on diverse hardware backends.

  • Emerging Languages/Platforms (Mojo):

    New programming paradigms like Mojo are being developed with the explicit goal of providing a high-performance, Python-syntax-compatible language for programming heterogeneous AI hardware, including GPUs and accelerators from various vendors.

    If successful, Mojo could offer a fundamentally different approach to AI software development, potentially bypassing some limitations of existing C++-based alternatives or framework-specific backends.

  • Direct PTX Programming (DeepSeek Example):

    The case of DeepSeek AI utilizing PTX directly on NVIDIA H800 GPUs to achieve highly efficient training for their 671B parameter MoE model demonstrates an extreme optimization strategy.

    By bypassing standard CUDA libraries and writing closer to the hardware's instruction set, they reportedly achieved significant efficiency gains.

    This highlights that even within the NVIDIA ecosystem, CUDA itself may not represent the absolute performance ceiling for sophisticated users willing to tackle extreme complexity, though it remains far beyond the reach of typical development workflows.

3.4 Implications for Pre-training Beyond CUDA

The pre-training landscape, while still dominated by NVIDIA, is showing signs of diversification, driven by cost pressures and strategic initiatives from competitors and hyperscalers. However, several factors shape the trajectory. Firstly, the sheer computational scale of pre-training necessitates high-end, specialized hardware. This means the battleground for pre-training beyond CUDA is primarily contested among major silicon vendors (NVIDIA, AMD, Intel, Google) and potentially large hyperscalers with custom chip programs, rather than being open to a wide array of lower-end hardware.

Secondly, software maturity remains the most significant bottleneck for alternative hardware platforms in the pre-training domain. While hardware like AMD Instinct and Intel Gaudi offer compelling specifications and cost advantages , their corresponding software stacks (ROCm, oneAPI/SynapseAI) are consistently perceived as less mature, stable, or easy to deploy at scale compared to the battle-hardened CUDA ecosystem. For expensive, long-duration pre-training runs where failures can be catastrophic, the proven reliability of CUDA often outweighs the potential benefits of alternatives, hindering faster adoption.

Thirdly, the reliance on high-level frameworks like PyTorch and JAX makes robust backend integration paramount. Developers interact primarily through these frameworks, meaning the success of non-NVIDIA hardware hinges less on the intricacies of ROCm or SYCL syntax itself, and more on the seamlessness, performance, and feature completeness of the framework's support for that hardware. This elevates the strategic importance of compiler technologies like Triton and XLA, which are responsible for translating framework operations into efficient machine code for diverse targets. Vendors must ensure their hardware is a first-class citizen within these framework ecosystems to compete effectively in pre-training.

4. Post-training Beyond CUDA: Optimization and Fine-tuning

4.1 Importance of Post-training

Once a large AI model has been pre-trained, further steps are typically required before it can be effectively deployed in real-world applications. These post-training processes include optimization – techniques to reduce the model's size, decrease inference latency, and improve computational efficiency – and fine-tuning – adapting the general-purpose pre-trained model to perform well on specific downstream tasks or datasets. These stages often have different computational profiles and requirements compared to the massive scale of pre-training, potentially opening the door to a broader range of hardware and software solutions.

4.2 Techniques and Tools Outside the CUDA Ecosystem

Several techniques and toolkits facilitate post-training optimization and fine-tuning on non-NVIDIA hardware:

  • Model Quantization: Quantization is a widely used optimization technique that reduces the numerical precision of model weights and activations (e.g., from 32-bit floating-point (FP32) to 8-bit integer (INT8) or even lower). This significantly shrinks the model's memory footprint and often accelerates inference speed, particularly on hardware with specialized support for lower-precision arithmetic.

    • OpenVINO NNCF:

      Intel's OpenVINO toolkit includes the Neural Network Compression Framework (NNCF), a Python package offering various optimization algorithms.

      NNCF supports post-training quantization (PTQ), which optimizes a model after training without requiring retraining, making it relatively easy to apply but potentially causing some accuracy degradation.

      It also supports quantization-aware training (QAT), which incorporates quantization into the training or fine-tuning process itself, typically yielding better accuracy than PTQ at the cost of requiring additional training data and computation.

      NNCF can process models from various formats (OpenVINO IR, PyTorch, ONNX, TensorFlow) and targets deployment on Intel hardware (CPUs, integrated GPUs, discrete GPUs, VPUs) via the OpenVINO runtime.

    • Other Approaches:

      While less explicitly detailed for ROCm or oneAPI in the provided materials, quantization capabilities are often integrated within AI frameworks themselves or through supporting libraries. The BitsandBytes library, known for enabling quantization techniques like QLoRA, recently added experimental multi-backend support, potentially enabling its use on AMD and Intel GPUs beyond CUDA.

      Frameworks running on ROCm or oneAPI backends might leverage underlying hardware support for lower precisions.

  • Pruning and Compression: Model pruning involves removing redundant weights or connections within the neural network to reduce its size and computational cost. NNCF also provides methods for structured and unstructured pruning, which can be applied during training or fine-tuning.

  • Fine-tuning Frameworks on ROCm/oneAPI: Fine-tuning typically utilizes the same high-level AI frameworks employed during pre-training, such as PyTorch, TensorFlow, or JAX, along with libraries like Hugging Face Transformers and PEFT (Parameter-Efficient Fine-Tuning).

    • ROCm Example:

      The process of fine-tuning LLMs using techniques like LoRA (Low-Rank Adaptation) on AMD GPUs via ROCm is documented.

      Examples demonstrate using PyTorch, the Hugging Face

      transformers
      

      library, and

      peft
      

      with the

      SFTTrainer
      

      on ROCm-supported hardware, indicating that standard parameter-efficient fine-tuning workflows can be executed within the ROCm ecosystem.

    • Intel Platforms:

      Fine-tuning can also be performed on Intel hardware, such as Gaudi accelerators

      or potentially GPUs supported by oneAPI, leveraging the respective framework integrations.

      The choice of hardware depends on the scale of the fine-tuning task.

  • Role of Hugging Face Optimum: Libraries like Hugging Face Optimum, particularly Optimum Intel, play a crucial role in simplifying the post-training workflow. Optimum Intel integrates OpenVINO and NNCF capabilities directly into the Hugging Face ecosystem, allowing users to easily optimize and quantize models from the Hugging Face Hub for deployment on Intel hardware. This integration streamlines the process for developers already working within the popular Hugging Face environment.

4.3 Hardware Considerations for Efficient Post-training

Unlike pre-training, which often necessitates clusters of the most powerful and expensive accelerators, fine-tuning and optimization tasks can sometimes be accomplished effectively on a wider range of hardware. Depending on the size of the model being fine-tuned and the specific task, single high-end GPUs (including professional or even consumer-grade NVIDIA or AMD cards ), Intel Gaudi accelerators , or potentially even powerful multi-core CPUs might suffice. This broader hardware compatibility increases the potential applicability of non-NVIDIA solutions in the post-training phase.

4.4 Implications for Post-training Beyond CUDA

The post-training stage presents distinct opportunities and challenges for CUDA alternatives. A key observation is the apparent strength of Intel's OpenVINO ecosystem in the optimization domain. The detailed documentation and tooling around NNCF for quantization and pruning provide a relatively mature pathway for optimizing models specifically for Intel's diverse hardware portfolio (CPU, iGPU, dGPU, VPU). This specialized toolkit gives Intel a potential advantage over AMD in this specific phase, as ROCm's dedicated optimization tooling appears less emphasized in the provided research beyond its core framework support.

Furthermore, the success of fine-tuning on alternative platforms like ROCm hinges critically on the robustness and feature completeness of the framework backends. As demonstrated by the LoRA example on ROCm, fine-tuning workflows rely directly on the stability and capabilities of the PyTorch (or other framework) implementation for that specific hardware. Any deficiencies in the ROCm or oneAPI backends will directly impede efficient fine-tuning, reinforcing the idea that mature software support is as crucial as raw hardware power.

Finally, there is a clear trend towards integrating optimization techniques directly into higher-level frameworks and libraries, exemplified by Hugging Face Optimum Intel. This suggests that developers may increasingly prefer using these integrated tools within their familiar framework environments rather than engaging with standalone, vendor-specific optimization toolkits. This trend further underscores the strategic importance for hardware vendors to ensure seamless and performant integration of their platforms and optimization capabilities within the dominant AI frameworks.

Table 4.1: Non-CUDA Model Optimization & Fine-tuning Tools

Tool/Platform Key Techniques Target Hardware Supported Frameworks/Formats Ease of Use/Maturity (Qualitative)
OpenVINO NNCF PTQ, QAT, Pruning (Structured/Unstructured) Intel CPU, iGPU, dGPU, VPU OpenVINO IR, PyTorch, TF, ONNX Relatively mature and well-documented for Intel ecosystem; Integrated with HF Optimum Intel
ROCm + PyTorch/PEFT Fine-tuning (e.g., LoRA, Full FT) AMD GPUs (Instinct, Radeon) PyTorch, HF Transformers Relies on ROCm backend maturity for PyTorch; Examples exist, but ecosystem maturity concerns remain
oneAPI Libraries Likely includes optimization/quantization libraries (details limited in snippets) Intel CPU, GPU, Gaudi PyTorch, TF (via framework integration) Aims for unified model, but specific optimization tool maturity less clear from snippets compared to NNCF
BitsandBytes (Multi-backend) Quantization (e.g., for QLoRA) NVIDIA, AMD, Intel (Experimental) PyTorch Experimental support for non-NVIDIA; Requires specific installation/compilation
Intel Gaudi + SynapseAI Fine-tuning Intel Gaudi Accelerators PyTorch, TF (via SynapseAI) Specific stack for Gaudi; Maturity relative to CUDA less established

5. Inference Beyond CUDA

5.1 The Inference Landscape: Diversity and Optimization

The inference stage, where trained and optimized models are deployed to make predictions on new data, presents a significantly different set of requirements compared to training. While training often prioritizes raw throughput and the ability to handle massive datasets and models, inference deployment frequently emphasizes low latency, high throughput for concurrent requests, cost-effectiveness, and power efficiency. This diverse set of optimization goals leads to a wider variety of hardware platforms and software solutions being employed for inference, creating more opportunities for non-NVIDIA technologies.

5.2 Diverse Hardware for Deployment

The hardware landscape for AI inference is notably heterogeneous:

  • CPUs & Integrated GPUs (Intel):

    Standard CPUs and the integrated GPUs found in many systems (particularly from Intel) are common inference targets, especially when cost and accessibility are key factors. Toolkits like Intel's OpenVINO are specifically designed to optimize model execution on this widely available hardware.

  • Dedicated Inference Chips (ASICs):

    Application-Specific Integrated Circuits (ASICs) designed explicitly for inference offer high performance and power efficiency for specific types of neural network operations. Examples include AWS Inferentia

    and Google TPUs (which are also used for inference).

  • FPGAs (Field-Programmable Gate Arrays):

    FPGAs offer hardware reprogrammability, providing flexibility and potentially very low latency for certain inference tasks. They can be adapted to specific model architectures and evolving requirements.

  • Edge Devices & NPUs:

    The proliferation of AI at the edge (in devices like smartphones, cameras, vehicles, and IoT sensors) drives demand for efficient inference on resource-constrained hardware.

    This often involves specialized Neural Processing Units (NPUs) or optimized software running on low-power CPUs or GPUs. Intel's Movidius Vision Processing Units (VPUs), accessible via OpenVINO, are an example of such edge-focused hardware.

  • AMD/Intel Data Center & Consumer GPUs:

    Data center GPUs from AMD (Instinct series) and Intel (Data Center GPU Max Series), as well as consumer-grade GPUs (AMD Radeon, Intel Arc), are also viable platforms for inference workloads.

    Software support comes via ROCm, oneAPI, or cross-platform runtimes like OpenVINO and ONNX Runtime.

5.3 Software Frameworks and Inference Servers

Deploying models efficiently requires specialized software frameworks and servers:

  • OpenVINO Toolkit & Model Server:

    Intel's OpenVINO plays a significant role in the non-CUDA inference space. It provides tools (like NNCF) to optimize models trained in various frameworks and a runtime engine to execute these optimized models efficiently across Intel's hardware portfolio (CPU, iGPU, dGPU, VPU).

    OpenVINO also integrates with ONNX Runtime as an execution provider

    and potentially offers its own Model Server for deployment.

    While some commentary questions its popularity relative to alternatives like Triton

    , it provides a clear path for inference on Intel hardware.

  • ROCm Inference Libraries (MIGraphX):

    AMD provides inference optimization libraries within the ROCm ecosystem, such as MIGraphX. These likely function as compilation targets or backends for higher-level frameworks or standardized runtimes like ONNX Runtime when deploying on AMD GPUs.

  • ONNX Runtime:

    The Open Neural Network Exchange (ONNX) format and its corresponding ONNX Runtime engine are crucial enablers of cross-platform inference. ONNX Runtime acts as an abstraction layer, allowing models trained in frameworks like PyTorch or TensorFlow and exported to the ONNX format to be executed on a wide variety of hardware backends through its Execution Provider (EP) interface.

    Supported EPs include CUDA, TensorRT (NVIDIA), OpenVINO (Intel), ROCm (AMD), DirectML (Windows), CPU, and others.

    This significantly enhances model portability beyond the confines of a single vendor's ecosystem.

  • NVIDIA Triton Inference Server:

    While developed by NVIDIA, Triton is an open-source inference server designed for flexibility.

    It supports multiple model formats (TensorRT, TensorFlow, PyTorch, ONNX) and backends (including OpenVINO, Python custom backends, ONNX Runtime).

    This architecture theoretically allows Triton to serve models using non-CUDA backends if appropriately configured.

    There is active discussion and development work on enabling backends like ROCm (via ONNX Runtime) for Triton

    , which could further position it as a more hardware-agnostic serving solution. However, its primary adoption and optimization focus remain heavily associated with NVIDIA GPUs.

  • Alternatives/Complements to NVIDIA Triton:

    The inference serving landscape includes several other solutions. vLLM has emerged as a highly optimized library specifically for LLM inference, utilizing techniques like PagedAttention and Continuous Batching, and reportedly offering better throughput and latency than Triton in some LLM scenarios.

    Other options include Kubernetes-native solutions like KServe (formerly KFServing), framework-specific servers like TensorFlow Serving and TorchServe, and integrated cloud provider platforms such as Amazon SageMaker Inference Endpoints

    and Google Vertex AI Prediction.

    The choice often depends on the specific model type (e.g., LLM vs. vision), performance requirements, scalability needs, and existing infrastructure.

  • DirectML (Microsoft):

    For Windows environments, DirectML provides a hardware-accelerated API for machine learning that leverages DirectX 12. It can be accessed via ONNX Runtime or other frameworks and supports hardware from multiple vendors, including Intel and AMD, offering another path for non-CUDA acceleration on Windows.

5.4 Implications for Inference Beyond CUDA

The inference stage represents the most fragmented and diverse part of the AI workflow, offering the most significant immediate opportunities for solutions beyond CUDA. The varied hardware targets and optimization priorities (cost, power, latency) create numerous niches where NVIDIA's high-performance, CUDA-centric approach may not be the optimal or only solution. Toolkits explicitly designed for heterogeneity, like OpenVINO and ONNX Runtime, are pivotal in enabling this diversification.

OpenVINO, in particular, provides a mature and well-defined pathway for optimizing and deploying models efficiently on the vast installed base of Intel CPUs and integrated graphics, making AI inference accessible without requiring specialized accelerators. ONNX Runtime acts as a crucial interoperability layer, effectively serving as a universal translator that allows models developed in one framework to run on hardware supported by another vendor's backend (ROCm, OpenVINO, DirectML, etc.). The adoption and continued development of these two technologies significantly lower the barrier for deploying models outside the traditional CUDA/TensorRT stack.

While NVIDIA's Triton Inference Server is powerful and widely used, its potential as a truly hardware-agnostic server remains partially realized. Although its architecture supports multiple backends, including non-CUDA ones like OpenVINO and ONNX Runtime , its primary association, optimization efforts, and community focus are still heavily centered around NVIDIA GPUs and the TensorRT backend. The active exploration of alternatives like vLLM for specific workloads (LLMs) and the ongoing efforts to add robust support for other backends like ROCm suggest that the market perceives a need for solutions beyond what Triton currently offers optimally for non-NVIDIA or highly specialized use cases.

Table 5.1: Inference Solutions Beyond CUDA

Solution (Hardware + Software Stack/Server) Target Use Case Key Features/Optimizations Framework/Format Compatibility Relative Performance/Cost Indicator (Qualitative)
Intel CPU/iGPU + OpenVINO Edge, Client, Cost-sensitive Cloud PTQ/QAT (NNCF), Latency/Throughput modes, Auto-batching, Optimized for Intel Arch OpenVINO IR, ONNX, TF, PyTorch Lower cost, wide availability; Performance depends heavily on CPU/iGPU generation and optimization
AMD GPU + ROCm / ONNX Runtime Cloud, Workstation Inference MIGraphX optimization, HIP, ONNX Runtime ROCm EP ONNX, PyTorch, TF (via ROCm backend) Potential cost savings vs NVIDIA; Performance dependent on GPU tier and ROCm maturity
Intel dGPU/VPU + OpenVINO Edge AI, Visual Inference Optimized for Intel dGPU/VPU hardware, Leverages NNCF OpenVINO IR, ONNX, TF, PyTorch Power-efficient options for edge; Performance competitive in target niches
AWS Inferentia + Neuron SDK Cloud Inference (AWS) ASIC optimized for inference, Low cost per inference, Neuron SDK compiler TF, PyTorch, MXNet, ONNX High throughput, low cost on AWS; Limited to AWS environment
Generic CPU/GPU + ONNX Runtime Cross-platform deployment Hardware abstraction via Execution Providers (CPU, OpenVINO, ROCm, DirectML, etc.) ONNX (from TF, PyTorch, etc.) Highly portable; Performance varies significantly based on chosen EP and underlying hardware
NVIDIA/AMD GPU + vLLM High-throughput LLM Inference PagedAttention, Continuous Batching, Optimized Kernels PyTorch (HF Models) Potentially higher LLM throughput/lower latency than Triton in some cases; Primarily GPU-focused
FPGA + Custom Runtime Ultra-low latency, Specialized tasks Hardware reconfigurability, Optimized data paths Custom / Specific formats Very low latency possible; Higher development effort, niche applications
Windows Hardware + DirectML / ONNX Runtime Windows-based applications Hardware acceleration via DirectX 12 API, Supports Intel/AMD/NVIDIA ONNX, Frameworks with DirectML support Leverages existing Windows hardware acceleration; Performance varies with GPU

6. Industry Outlook and Conclusion

6.1 Market Snapshot: Current Share and Growth Trends

The AI hardware market, particularly for data center compute, remains heavily dominated by NVIDIA. Current estimates place NVIDIA's market share for AI accelerators and data center GPUs in the 80% to 92% range. Despite this dominance, competitors are present and making some inroads. AMD has seen its data center GPU share grow slightly, reaching approximately 4% in 2024, driven by adoption from major cloud providers. Other players like Huawei hold smaller shares (around 2% ), and Intel aims to capture market segments with its Gaudi accelerators and broader oneAPI strategy.

The overall market is experiencing explosive growth. Projections for the AI server hardware market suggest growth from around $157 billion in 2024 to potentially trillions by the early 2030s, with a compound annual growth rate (CAGR) estimated around 38%. Similarly, the AI data center market is projected to grow from roughly $14 billion in 2024 at a CAGR of over 28% through 2030. The broader AI chip market is forecast to surpass $300 billion by 2030. Within these markets, GPUs remain the dominant hardware component for AI , inference workloads constitute the largest function segment , cloud deployment leads over on-premises , and North America is the largest geographical market.

6.2 Adoption Progress and Remaining Hurdles for CUDA Alternatives

Significant efforts are underway to build viable alternatives to the CUDA ecosystem. AMD's ROCm has matured, gaining crucial support within PyTorch and securing design wins with hyperscalers for its Instinct accelerators. Intel's oneAPI offers a comprehensive vision for heterogeneous computing backed by the UXL Foundation, and its OpenVINO toolkit provides a strong solution for inference optimization and deployment on Intel hardware. Abstraction layers and compilers like PyTorch 2.0, OpenAI Triton, and OpenXLA are evolving to provide more hardware flexibility.

Despite this progress, substantial hurdles remain for widespread adoption of CUDA alternatives. The primary challenge continues to be the maturity, stability, performance consistency, and breadth of the software ecosystems compared to CUDA. Developers often face a steeper learning curve, more complex debugging, and potential performance gaps when moving away from the well-trodden CUDA path. The sheer inertia of existing CUDA codebases and developer familiarity creates significant resistance to change. Furthermore, the alternative landscape is fragmented, lacking a single, unified competitor to CUDA, which can dilute efforts and slow adoption. While the high cost of NVIDIA hardware is a strong motivator for exploring alternatives , these software and ecosystem challenges often temper the speed of transition, especially for mission-critical training workloads.

6.3 The Future Trajectory: Towards a More Heterogeneous AI Compute Landscape?

The future of AI compute appears poised for increased heterogeneity, although the pace and extent of this shift remain subject to competing forces. On one hand, NVIDIA continues to innovate aggressively, launching new architectures like Blackwell, expanding its CUDA-X libraries, and building comprehensive platforms like DGX systems and NVIDIA AI Enterprise. Its deep ecosystem integration and performance leadership, particularly in high-end training, provide a strong defense for its market share.

On the other hand, the industry push towards openness, cost reduction, and strategic diversification is undeniable. Events like the Beyond CUDA Summit , initiatives like the AI Alliance (including AMD, Intel, Meta, etc. ), the UXL Foundation , and the significant investments by hyperscalers in custom silicon or alternative suppliers all signal a concerted effort to reduce dependence on NVIDIA's proprietary stack. Geopolitical factors and supply chain vulnerabilities, particularly the heavy reliance on TSMC for cutting-edge chip manufacturing, also represent potential risks for NVIDIA's long-term dominance and could further incentivize diversification.

The most likely trajectory involves a gradual diversification, particularly noticeable in the inference space where hardware requirements are more varied and cost/power efficiency are paramount. Toolkits like OpenVINO and runtimes like ONNX Runtime will continue to facilitate deployment on non-NVIDIA hardware. In training, while NVIDIA is expected to retain its lead in the highest-performance segments in the near term, competitors like AMD and Intel will likely continue to gain share, especially among cost-sensitive enterprises and hyperscalers actively seeking alternatives. The success of emerging programming models like Mojo could also influence the landscape if they gain significant traction.

6.4 Concluding Remarks on the Viability and Impact of the "Beyond CUDA" Movement

Synthesizing the analysis of the GTC video's focus on compute beyond CUDA and the broader industry research, it is clear that NVIDIA's CUDA moat remains formidable. Its strength lies not just in performant hardware but, more critically, in the deeply entrenched software ecosystem, developer inertia, and high switching costs accumulated over nearly two decades. Overcoming this requires more than just competitive silicon; it demands mature, stable, easy-to-use software stacks, seamless integration with dominant AI frameworks, and compelling value propositions.

However, the "Beyond CUDA" movement is not merely aspirational; it is a tangible trend driven by significant investment, strategic necessity, and a growing ecosystem of alternatives. Progress is evident across hardware (AMD Instinct, Intel Gaudi, TPUs, custom silicon) and software (ROCm, oneAPI, OpenVINO, Triton, PyTorch 2.0, ONNX Runtime). While a complete upheaval of CUDA's dominance appears unlikely in the immediate future, the landscape is undeniably shifting towards greater heterogeneity. Inference deployment is diversifying rapidly, and competition in the training space is intensifying. The ultimate pace and extent of this transition will depend crucially on the continued maturation and convergence of alternative software ecosystems and their ability to provide a developer experience and performance level that can effectively challenge the CUDA incumbency. The coming years will be critical in determining whether the industry successfully cultivates a truly multi-vendor, open, and competitive AI compute environment.

CXLMemUring Update

1. 背景:CXL 与内存体系的演进

随着 CXL(Compute Express Link)对外设带宽和内存一致性需求的支持不断升级,越来越多的场景(如高性能计算、数据中心、AI 加速等)开始采用 CXL 的缓存一致性特性(CXL.cache)来让外部设备更灵活地访问主机内存。然而,这也带来了新的挑战:

设备的热插拔与故障风险

在传统的 PCIe/CXL.io 模式下,如果设备被热拔或因软硬件错误停机,通常只影响到 I/O 操作;但在 CXL.cache 模式下,外设有可能持有“Modified”状态的缓存行,一旦设备意外退出,则无法安全地将这些缓存行同步回主内存,可能导致数据丢失或更严重的系统级错误(如内核崩溃)。
安全性与侧信道风险

CXL.cache 在带来跨设备缓存一致性的同时,也暴露了更细粒度的访问通道。以往仅在 CPU 内部可能出现的缓存侧信道,如今可能演变为跨设备的安全隐患。
传统的防护方法(如粗粒度的地址访问控制)难以及时而精细地阻断潜在的恶意访问或侧信道攻击。
同步性能与延迟

随着更多设备加入一致性域,诸如共享队列、分布式归约(reduce)操作等场景的内存同步开销会显著上升。硬件需要更多手段来降低同步延迟。

Von Neumann Memory Wall 计算和存储之间的带宽与延迟矛盾依旧是系统瓶颈。即便有了 CXL.cache,对外设的访问在延迟和带宽方面仍难与 CPU 内部缓存层级相媲美。
基于这些问题,业界和学术界都在探索新的硬件/软件协作方案,以便在保证安全性的同时提升可靠性和性能。

2. 现有 SoTA(State-of-The-Art)方案及不足

2.1 现有典型方案
单向或异步加载/存储

通过在外设或内存侧增加多级缓冲或流水线机制,以期减少 CPU 对外设频繁通信时的阻塞。
在高性能计算或数据中心场景下,也有通过大页(Huge Page)、NUMA 优化等方法,减轻 CPU-设备间的通讯负担。
数据流加速器(Data Streaming Accelerator)

在 GPU、FPGA 或定制加速器中,往往引入类似 SPDK、DSA(Intel Data Streaming Accelerator)这样的技术,以批处理方式提升大吞吐数据操作的效率,减少内核干预。
大页或段级别的防护/隔离

在内存管理层面,通过页表标记等方式提供一定程度的访问隔离,把外设可访问的地址范围限制在特定分区/页中,避免无关的数据泄露。
2.2 现有方案的局限
Cache 污染与侧信道仍难彻底解决

即使采用大页或统一缓存一致性,外设在与 CPU 共享部分 LLC(Last-Level Cache)时,仍可能带来新的共享缓存侧信道。
粗粒度的防护手段难以阻止细粒度的攻击。
设备断开时的数据一致性问题

CXL.cache 模式下,如果关键数据一直处于外设的缓存行中(Modified 状态),当设备意外下线,将导致数据无法回写或者系统出现一致性错误。
目前缺乏一种硬件级的快速回退(rollback)或事务式保护机制。
同步操作的开销高

随着分布式、并行化的计算模式增多,共享数据结构(如环形队列、归约缓冲区等)的访问需要更多内存屏障或锁操作;跨设备的锁与屏障机制往往造成显著的延迟堆积。
编程与调试难度大

现有体系需要在设备驱动、内核、应用层多处改动;开发者若想在总线或内存级别做更灵活、更细粒度的管理,需要投入大量复杂的硬件/软件配合。

3. 新思路:在内存总线增加 eBPF-like 指令支持

为同时兼顾可靠性、安全性和性能,我们提出在主机与 CXL/root complex 之间增设一个“微型 eBPF-like 指令处理单元(tiny eBPF CPU)”的方案,核心思想如下:

总线级事务式内存操作(Transactional Memory on Bus)

允许外设将一组对关键控制面的内存写操作打包(类似 eBPF 程序),由总线侧的小型处理器执行并保证事务性。
如果事务执行中途外设意外断连或故障,整个事务可以回滚,避免出现“Modified”缓存行无人回写的情况。
这样一来,外设本身不再直接持有关键数据的独占/修改状态,也降低了热插拔带来的数据丢失风险。
可编程的细粒度安全检查

通过在总线侧维护一个可编程匹配表(matching table),结合 eBPF-like 指令,可以根据地址、访问模式、数据模式等进行快速检查和过滤;
实现类似 Linux seccomp 的“白名单”或“黑名单”机制,在更细的缓存行或地址范围上阻断可疑访问,减轻潜在侧信道攻击的风险。
降低同步延迟

由于部分跨设备的原子操作、锁或屏障可在总线侧以事务性执行,某些情况下比传统 DMA 往返更高效;
减少了软件层对外设状态的一致性管理负担,整体提升系统的并发性能。
灵活的可扩展性

eBPF-like 指令可以进一步应用于性能分析或诊断场景,比如在总线层监听和统计特定访问模式,以帮助开发者或系统管理员了解数据流分布,定位瓶颈或安全风险。

4. 实现策略

4.1 硬件结构与可能路径
CXL 专用实现

在 CXL Root Complex 侧增加一个微处理器核,用于执行 eBPF-like 指令包;
提供专门的 side-band 通道给外设发送这些指令包;
需在 CXL 规范中扩展或定义新协议字段,以实现对 CXL.cache 设备的安全事务化管理。
复用现有 NoC/总线协议

在诸如 TileLink、AMBA-AXI/ACE/CHI 等片上网络或总线协议中,加入 side-band 通道来传输 eBPF-like 指令;
好处是可以借助开源 IP 或既有总线生态,易于原型验证;
挑战在于,这些总线方案主要针对片上或极短距离的高带宽互连,如何说服审稿人或业界接受其“跨芯片”或外设场景,需要进一步探讨。
RISCV 扩展与软件评估

在 RISC-V 中扩展 LR/SC 指令序列,让 CPU 或总线能够将这些指令序列包装成 eBPF-like 程序并交给硬件执行;
有利于快速在软件层验证概念,可构建模拟器或 FPGA 测试平台来评估性能增益和安全增强效果。
4.2 关键设计点
事务内存 vs. eBPF

事务内存主要提供原子性和一致性保障;
eBPF-like 部分提供灵活的编程接口,用于地址匹配、安全策略执行、数据过滤等。
二者结合使得硬件具备高可编程性与安全回退的能力。
安全防护与可控回退

可在总线侧设置专门的访问策略:
对关键地址范围强制使用事务模式;
若外设故障,回退该事务并拒绝后续访问;
由此杜绝“Modified”缓存行无法回写的情况。
性能与兼容性

需要平衡新硬件引入的额外时延(如指令解析、表项匹配等)与带来的安全/可控好处;
对于普通访问,可走传统路径,不额外产生开销;
对需要安全/事务保障的访问,才使用 eBPF-like 处理器,减少对整体系统吞吐的影响。

5. 评估与展望

安全与Profiling演示

在小规模实验或原型系统中,通过 eBPF-like 指令完成地址过滤、访问统计,以及在外设异常时触发回退和阻断;
以此证明该方案能有效应对细粒度的安全风险。
事务内存的可行性

对比传统软件锁或 TSX(Intel Transactional Synchronization Extensions)等方式,验证在跨设备场景下,硬件级回退如何避免外设故障带来的数据丢失;
探讨如何防御 TSX 异步中止(Asynchronous Abort)等可能的攻击手段。
对现行 CXL Type 1/Type 2/Type 3 设备的兼容

如果只面向 CXL Type 2(带缓存一致性的设备),则需要在设计上确保对传统纯内存类设备(Type 3)或无缓存一致性的设备也能部分兼容;
在不同类型设备上分别测试性能和安全收益,形成全面的评估报告。

6. 总结

面对未来异构计算与内存层次日益复杂的趋势,“在内存总线增加 eBPF-like 指令支持”是一种兼顾安全性、可靠性及性能的前瞻性方案。它通过在总线侧对关键地址访问进行事务化处理并融合可编程逻辑,实现了:

避免设备热插拔和故障所引发的关键数据丢失;
在更精细的粒度上进行安全访问控制,减轻侧信道风险;
减少软件同步开销,提高分布式或并行场景下的整体效率。
尽管实现过程中需要对现行 CXL 或其他总线协议做出一定扩展,但从概念验证(PoC)到实际硬件落地,都具备可行的技术路线与潜在价值。后续仍需在事务内存模型、eBPF-like 指令的语义及安全性加固策略上持续探索与完善。

参考文献

  1. Fan Yao, et al. “CACo: A Comprehensive Approach to Cache Covert Channel Mitigation”, HPCA, 2018.

ChinaSys 2024 Fall








AI Chip



TrEnv














堆扩展不行









remote fork 容器本身。


SIGMETRICS





变长图


低比特量化


visually






like omnitable

Skyloft



NeoMem











failure durability
type 2 accelerator.
PTX JIT is better than NVBit.
















undefined reference to `__sync_fetch_and_add_4'

You couldn't export builtin functions, but you can do hack with gcc

gcc -shared -fPIC -O2 atomic_ops.c -o libatomic_ops.so
#include <stdint.h>
#include <stdbool.h>

// Ensure all functions are exported from the shared library
#define EXPORT __attribute__((visibility("default")))

// 32-bit compare and swap
EXPORT
bool __sync_bool_compare_and_swap_4(volatile void* ptr, uint32_t oldval, uint32_t newval) {
    bool result;
    __asm__ __volatile__(
        "lock; cmpxchgl %2, %1\n\t"
        "sete %0"
        : "=q" (result), "+m" (*(volatile uint32_t*)ptr)
        : "r" (newval), "a" (oldval)
        : "memory", "cc"
    );
    return result;
}

// 64-bit compare and swap
EXPORT
bool __sync_bool_compare_and_swap_8(volatile void* ptr, uint64_t oldval, uint64_t newval) {
    bool result;
    __asm__ __volatile__(
        "lock; cmpxchgq %2, %1\n\t"
        "sete %0"
        : "=q" (result), "+m" (*(volatile uint64_t*)ptr)
        : "r" (newval), "a" (oldval)
        : "memory", "cc"
    );
    return result;
}

// 32-bit fetch and add
EXPORT
uint32_t __sync_fetch_and_add_4(volatile void* ptr, uint32_t value) {
    __asm__ __volatile__(
        "lock; xaddl %0, %1"
        : "+r" (value), "+m" (*(volatile uint32_t*)ptr)
        :
        : "memory"
    );
    return value;
}

// 32-bit fetch and or
EXPORT
uint32_t __sync_fetch_and_or_4(volatile void* ptr, uint32_t value) {
    uint32_t result, temp;
    __asm__ __volatile__(
        "1:\n\t"
        "movl %1, %0\n\t"
        "movl %0, %2\n\t"
        "orl %3, %2\n\t"
        "lock; cmpxchgl %2, %1\n\t"
        "jne 1b"
        : "=&a" (result), "+m" (*(volatile uint32_t*)ptr), "=&r" (temp)
        : "r" (value)
        : "memory", "cc"
    );
    return result;
}

// 32-bit val compare and swap
EXPORT
uint32_t __sync_val_compare_and_swap_4(volatile void* ptr, uint32_t oldval, uint32_t newval) {
    uint32_t result;
    __asm__ __volatile__(
        "lock; cmpxchgl %2, %1"
        : "=a" (result), "+m" (*(volatile uint32_t*)ptr)
        : "r" (newval), "0" (oldval)
        : "memory"
    );
    return result;
}

// 64-bit val compare and swap
EXPORT
uint64_t __sync_val_compare_and_swap_8(volatile void* ptr, uint64_t oldval, uint64_t newval) {
    uint64_t result;
    __asm__ __volatile__(
        "lock; cmpxchgq %2, %1"
        : "=a" (result), "+m" (*(volatile uint64_t*)ptr)
        : "r" (newval), "0" (oldval)
        : "memory"
    );
    return result;
}

// Additional commonly used atomic operations

// 32-bit atomic increment
EXPORT
uint32_t __sync_add_and_fetch_4(volatile void* ptr, uint32_t value) {
    uint32_t result;
    __asm__ __volatile__(
        "lock; xaddl %0, %1"
        : "=r" (result), "+m" (*(volatile uint32_t*)ptr)
        : "0" (value)
        : "memory"
    );
    return result + value;
}

// 32-bit atomic decrement
EXPORT
uint32_t __sync_sub_and_fetch_4(volatile void* ptr, uint32_t value) {
    return __sync_add_and_fetch_4(ptr, -value);
}

// 32-bit atomic AND
EXPORT
uint32_t __sync_fetch_and_and_4(volatile void* ptr, uint32_t value) {
    uint32_t result, temp;
    __asm__ __volatile__(
        "1:\n\t"
        "movl %1, %0\n\t"
        "movl %0, %2\n\t"
        "andl %3, %2\n\t"
        "lock; cmpxchgl %2, %1\n\t"
        "jne 1b"
        : "=&a" (result), "+m" (*(volatile uint32_t*)ptr), "=&r" (temp)
        : "r" (value)
        : "memory", "cc"
    );
    return result;
}

// 32-bit atomic XOR
EXPORT
uint32_t __sync_fetch_and_xor_4(volatile void* ptr, uint32_t value) {
    uint32_t result, temp;
    __asm__ __volatile__(
        "1:\n\t"
        "movl %1, %0\n\t"
        "movl %0, %2\n\t"
        "xorl %3, %2\n\t"
        "lock; cmpxchgl %2, %1\n\t"
        "jne 1b"
        : "=&a" (result), "+m" (*(volatile uint32_t*)ptr), "=&r" (temp)
        : "r" (value)
        : "memory", "cc"
    );
    return result;
}

2024 年终总结

各位朋友,大家好。时间过得非常快,转眼又到年终岁末,想借这个机会做一个比较详尽、回顾式的总结,也为自己接下来的学习和工作做一个展望。接下来这段总结可能会稍微长一点,大概会花上十分钟的时间跟大家分享一下我今年的经历、感悟以及对未来的期待,希望大家能耐心听我唠一唠。

一、身体与心态的转折

今年的五到十二月,对我来说是一段并不算轻松的时光。由于生病的关系,我被迫中断了不少科研与学习的进度,也暂时离开了我非常热爱的技术领域。在那段日子里,身体上的不适与心灵上的焦虑常常相互影响,一方面我担心康复是否能够顺利,另一方面也忧虑自己是否会因此错过技术社区的更新迭代,以及可能涌现的新机会。

然而,这段“被迫停下脚步”的时间也给了我一个宝贵的缓冲期,让我能够更加冷静地思考:到底自己为什么会对 eBPF、WASM、CXL 等方向投入这么多的热情?到底想在未来的科研或工业实践中扮演怎样的角色?我慢慢意识到,保持对世界的好奇、享受做科研和技术探索的乐趣,其实才是我内心深处最坚定的东西。这也让我对自己今后的人生布局有了更清晰的感受和方向——求真、探索和创造比起一时的成就和荣誉似乎更重要。

二、回顾与反思:从好奇心出发

我一直觉得,一个人能够持续走得远,最大的动力往往是源自内心深处的好奇心。在这一年里,虽然有一段时间我无法亲身投入各种项目和话题的第一线,但我一直在关注产业和学术界一些新的动态,也不时阅读各种新闻、博客和技术更新,让我在“被迫停下来”的期间依旧保持对行业的敏感度。

  • eBPF(Extended Berkeley Packet Filter)
    之所以对 eBPF 感兴趣,是因为它为内核态与用户态之间的灵活沟通提供了前所未有的机会。在内核层面,对网络、跟踪、可观测性、安全策略等的精细控制,让我看到操作系统领域新的可能性。尽管它背后的概念已经出现多年,但近几年的蓬勃发展证明了它在云原生时代和超大规模分布式系统中仍有很大的增长空间。
  • WASM(WebAssembly)
    WebAssembly 近几年从前端扩展到后端,甚至已经进入云计算和边缘计算的范畴。它高度跨平台、可移植、高效的特性,让我看到未来在容器化和函数计算层面的更多机会。我一直好奇 WASM 是否会在云端或边缘带来类似当年 Docker、Kubernetes 带来的生态变革,毕竟能够在各种硬件、各种语言环境下运行的可移植性,对整个行业来说是非常有吸引力的。
  • CXL(Compute Express Link)
    这是一个让存储与计算之间实现更紧密交互的新兴技术标准,尤其适用于数据中心、云计算以及对计算与存储性能要求极高的场景。CXL 提供了一种共享和统一管理内存或加速器资源的方式,让我对未来服务器架构和高性能计算的演进充满好奇。我想,如果 eBPF 和 WASM 关注的是“软件层面的革新”,那么 CXL 则是硬件层面的重大突破。有时候,把眼光放到系统堆栈的更底层,也能带来全新的灵感和思考。

这些领域都各自处于技术发展的前沿,也都有各自的挑战。我很喜欢这种“万花筒”般的世界:变化既是挑战,也是机遇,旧的知识会逐渐被新思路取代;新人只要肯投入时间和精力,就能找到可以施展的空间。这种迭代的活力深深地吸引着我,也让我一直想要在学习或工作中关注更广、更深的领域。

三、拥抱变化:科研、工业和开源

在回顾这一年、反思自身状态的时候,我发现自己心中始终有一个纠结:究竟是继续在学术界深耕,比如考虑读博士、做研究;还是去工业界寻找更直接的项目落地场景?亦或是投身开源社区,在一个大家庭里共同打磨一款产品?

其实,这三种方向并不一定相互排斥。正如许多成功的前沿技术往往源自学术研究,再经过开源社区的孵化,最后才在工业界大规模应用。我看到 eBPF、WASM、CXL 也在走着类似的路径:先由少数极客在学术或社区环境中推波助澜,逐渐积累,形成一定规模,再被大企业收购或引入到正式的生产环境。我个人非常希望能在这个过程中尽一份力,从早期就参与其中,并在学术和工业应用上都能获得成长与磨砺。

1. 学术探索

我对学术研究的热爱,更多是对于“未知领域的好奇”。学术研究可以让我沉下心来思考:技术背后更底层、更本质的问题在哪里?这样的问题往往并非一朝一夕就能解决,需要的是长时间的积累和深度思考。同时,研究的过程也能让我对事物有更全面和系统的认识,哪怕只是在科研群体的圈子里互相切磋,也能激发创新火花。

2. 工业场景

工业界的实践能带给我更多“真刀真枪”的挑战。如何解决海量数据带来的网络瓶颈与安全风险?如何提升系统性能、减少资源浪费?这些问题没有现成的标准答案,需要我们从需求出发,结合实际的工作负载和使用场景去做权衡。eBPF、WASM 和 CXL 在这一领域都大有可为:前者能在内核层实现灵活的策略定义与性能分析,后者可为跨平台运行与硬件加速提供新的方案。

3. 开源社区

我相信开源社区代表了一种更加开放、自由的创新方式。只要感兴趣,任何人都可以在全球范围内基于同一个代码库、文档、工具集做出贡献和探索。像 Cilium(基于 eBPF 的网络与安全),这些开源项目已经有了可观的商业价值,也吸引了许多开发者。我非常向往这种“你一行代码、我一行代码,大家一起把未来拼出来”的氛围,既能让技术传播更广,也能帮助个人在更短的时间里融入国际化的技术浪潮。

四、展望未来:持续好奇、持续前行

经历了生病的休整和对未来的沉淀思考,我在下半年重新回到科研与实践当中。我将主要从以下几个方面着手,继续深化与拓展:

  1. 深化对 eBPF 的理解与研究
    我会结合具体的场景,比如网络安全、可观测性、微服务治理等,探究 eBPF 在极端规模(大流量或复杂网络拓扑)下的优势和挑战。并且尝试与其他新兴技术结合,看是否能拓展更多的应用形态。
  2. 探索 WASM 在云端与边缘的应用
    我想亲手实验一些基于 WASM 的函数计算平台,把它们与容器技术进行横向对比,看看不同的部署环境、不同的语言绑定,会给性能、扩展性和可维护性带来怎样的差异,也希望在这个过程中能摸索出更优的实践方案。
  3. 关注 CXL 对新一代数据中心与 HPC 的影响
    虽然对硬件层面还不算太熟悉,但我会持续关注社区和行业的一些新动向,尽可能在底层架构优化或资源管理的角度上学习 CXL 能带来的新机遇。如果有机会,也希望与志同道合的团队一起做一些实验性的项目,验证 CXL 在实际应用中的价值和限制。
  4. 加强学术与工业界的连接
    我会继续思考是否要在学术研究上投入更多时间,比如考虑读博士,或者在高校、研究机构里担任研究助理;同时也可能会到工业界或者开源社区参与实习或短期项目。这些都是为了能够将理论与实践更好地结合,让研究问题与真实需求互相促进、共同进步。
  5. 保持对身体健康的关注
    病痛教会我,只有保持健康,我们才能够持续地投入到我们热爱的事情上。所以,在忙碌的科研与工作之外,我也会花更多的时间休息、运动、调节心态,让自己在追逐梦想的同时,不至于因为身体等因素而再次中断。

五、结语

回顾这一年,虽然从五月到十二月有相当长的一段时间被疾病困扰,但我觉得自己反而从中得到了更多关于人生与未来的启示。我更加明确:我真正的热爱与动力,还是源自对世界的好奇,对技术与创新的热情。无论未来做学术、做工业还是做开源,乃至于探索更多可能性,最重要的是不要失去那份“想玩的开心、想探索未知”的初心。

感谢大家在这一年里对我的关注、陪伴和帮助,也谢谢每一位给我提供技术指导和生活支持的朋友们。明年,希望我可以在 eBPF、WASM、CXL 等方向上取得更多进步,也期待能与更多有同样好奇心和热情的小伙伴们携手前行,在这个快速变化的时代,一起去创造新的可能性。

祝愿大家在新的一年身体健康、万事顺遂,也祝我们每个人都能继续保持对世界的好奇和探索的勇气。让我们一起迎接下一个更具挑战与机会的2025年!

谢谢大家!

C++ coroutine segfault for assigning into a shared object

struct Task {
    struct promise_type;
    using handle_type = std::coroutine_handle<promise_type>;

    struct promise_type {
        auto get_return_object() { 
            return Task{handle_type::from_promise(*this)}; 
        }
        std::suspend_never initial_suspend() { return {}; }
        std::suspend_always final_suspend() noexcept { return {}; }
        void return_void() { }
        void unhandled_exception() {}
    };

    handle_type handle;
    
    Task(handle_type h) : handle(h) {}
    ~Task() {
        if (handle) handle.destroy();
    }
    Task(const Task&) = delete;
    Task& operator=(const Task&) = delete;
    Task(Task&& other) : handle(other.handle) { other.handle = nullptr; }
    Task& operator=(Task&& other) {
        if (this != &other) {
            if (handle) handle.destroy();
            handle = other.handle;
            other.handle = nullptr;
        }
        return *this;
    }

    bool done() const { return handle.done(); }
    void resume() { handle.resume(); }
};
Task process_queue_item(int i) {
    if (!atomicQueue[i].valid) {
        co_await std::suspend_always{};
    }
    atomicQueue[i].res = remote1(atomicQueue[i].i, atomicQueue[i].a, atomicQueue[i].b);
}

why line atomicQueue[i].res = ... cause segfault?

Coroutine lifetime issues: If the coroutine is resumed after the atomicQueue or its elements have been destroyed, this would lead to accessing invalid memory.

solusion

Task process_queue_item(int i) {
    if (i < 0 || i >= atomicQueue.size()) {
        // Handle index out of bounds
        co_return;
    }
    
    if (!atomicQueue[i].valid) {
        co_await std::suspend_always{};
    }
    
    // Additional check after resuming
    if (!atomicQueue[i].valid) {
        // Handle unexpected invalid state
        co_return;
    }
    
    try {
        atomicQueue[i].res = remote1(atomicQueue[i].i, atomicQueue[i].a, atomicQueue[i].b);
    } catch (const std::exception& e) {
        // Handle any exceptions from remote1
        // Log error, set error state, etc.
    }
}

MOAT: Towards Safe BPF Kernel Extension

MPK only supports up to 16 domains, the # BPF could be way over this number. We use a 2-layer isolation scheme to support unlimited BPF programs. The first layer deploys MPK to set up a lightweight isolation between the kernel and BPF programs. Also, BPF helper function calls is not protected, and can be attacked.

  1. They use the 2 layer isolation with PCID. In the first layer, BPF Domain has protection key permission lifted by kernel to do corresponding work, only exception is GDT and IDT they are always write-disabled. The second layer, when a malicious BPF program tries to access the memory regions of another BPF program, a page fault occurs, and the malicious BPF program is immediately terminated. To avoid TLB flush, each BPF program has PCID and rarely overflow 4096 entries.

  1. helper: 1. protect sensitive objects It has critical object finer granularity protection to secure. 2. ensure the validity of the parameters. It(Dynamic Parameter Auditing (DPA)) leverages the information obtained from the BPF verifier to dynamically check if the parameters are within their legitimate ranges.

LibPreemptible

uintr come with Sapphire Rapids,(RISCV introduce N extension at 2019) meaning no context switches compared with signal, providing lowest IPC Latency. Using APIC will incur safety concern.

uintr usage

  1. general purpose IPC
  2. userspace scheduler(This paper)
  3. userspace network
  4. libevent & liburing

syscall addication(eventfd like) sender initiate and notify the event and receiver get the fd call into kernel and senduipi back to sender.

WPS Office 2024-07-27 20.44.35
They wrote a lightweight runtime for libpreemptible.

  1. enable lightweight and fine grained preemption
  2. Separation of mechanism and policy
  3. Scability
  4. Compatibility

They maintained a fine-grained(3us) and dynamic timers for scheduling rather than kernel timers. It can greatly improve the 99% tail latency. Normal design of SPR's hw feature.

Reference

  1. https://github.com/OS-F-4/qemu-tutorial/blob/master/qemu-tutorial.md