CSE290X - victoryang00’ blog

Introduction to CXL Type 1

Guided Usecase

[1] and [2] are just qemu's implementation of dm-crypto for LUKS, that every device mapper over a physical block device will require a key and a crypto accelerator/software crypto implementation to decrypt to get the data. We implement a crypto accelerator CXL type 1 semantic over a framework of virtio-crypto-pci. We want to emulate the mal state or unplug the crypto device; the kernel will get ATS bit DMAed data and will resume by CPU software crypto implementation.

Device emulation

DMA and access memory

Create a CacheMemRegion that maps a specific SPP region for one on one mapping of a bunch of CXL.cache caches on a CXL device.

Crypto operations

When calling crypto operations in the kernel, we actually offload the encrypt/decrypt operations to the type 1 accelerator through CXL.io, which tells the device cast on operation on arbitrary SPP. The accelerator will first take ownership of arbitrary SPP in the CacheMemRegion and notify the host. Eventually, the host will get the shared state of the SPP's cacheline.

Cache coherency emulation

struct D2HDataReq = {
    D2H DataHeader 24b;
    opcode 4b;
    CXL.cache Channel Crediting;
}
struct CXLCache = { 
    64Byte Data; 
    MESI 4 bit;
    ATS 64 bit;
    [D2HDataReq;n] remaining bit;
}

Metadata with Intel SPP write protection support. Mark the access to arbitrary cacheline to the SPP. We need to perform all the transaction descriptions and queuing in the 64Byte of the residue data in the SPP. The arbitrary operation according to the queue will cast on the effect of MESI bit change and update writes protection for the Sub Page and Root Complex or other side effects like switches change.

The host and device's request is not scheduled FIFO, but the host's seeing the data will have better priority. So the H2D req will be consumed first and do D2H FIFO. All the operations follow interface operation to CXLCache.

Taking exclusiveness-able

We mark the Transportation ATS bit be whether taking exclusiveness-able. and copy the cacheline in another map; once emulated unplugged, the cacheline is copied back for further operation of the kernel to resume software crypto calculation.

How to emulate the eviction

We have two proposals

qemu pebs to watch the cache evict of the physical address of an SPP
use sub-page pin page_get_fast to pin to a physical address within the last level cache. [7]

Tag: CSE290X

Qemu CXL type 1 emulation proposal