In , the author talked about the Asynchronous Memory Unit that the CPU and Memory controller needs to support of co-design.
The overhead of hardware consistency checking is one reason that limits the capacity of traditional load/store queues and MSHRs. The AMU leaves the consistency issue to the software. They argue that software and hardware cooperation is the right way to exploit the memory parallelism over large latency for AMU.
As shown in the Figure of sensitivity tests in , the decomposition analysis of DirectCXL shows a completely different result: no software and no data copy overhead. As the payload increases, the main component of the DirectCXL latency is the LLC (CPU Cache). This is because the Miss State Holding Register (MSHR) in the CPU LLC can handle 16 concurrent misses, so with large payload data, many memory requests (64B) are suspended on the CPU, and processing a 4KB payload takes up 67% of the total latency.
The conclusion is MHSR inside the CPU is not enough to deal with memory load in the CXL.mem world, and both the latency and the bandwidth are so diverse across the serial PCIe5 lane. Also, another possible outcome compared with RDMA SRQ approach of the controller, we think the PMU and semantics of coherency still matter and the future way of persistency according to the Huawei's approach and SRQ approaches will fall back to ld/st but with a smarter leverage in the MC that asynchronously ld/st the data.
Asynchronous memory access unit for general purpose processors
Direct Access, High-Performance Memory Disaggregation with DirectCXL
MMAP a huge file need OS to register a virtual address to mmap the file on; once any request to the file is made, we may use page fault to load the file from disk to the private DRAM and setup the va_to_pa and buffer the file part in the DRAM, maybe use TLB to cache the next read. Every CXL device has it own mapping of memory; if you MMAP memory that was swapped onto CXL.mem devices like memory semantic SSD, the controller of SSD may decide whether to put on on-SSD DRAM or SSD and, in the backend, write through everything on physical media. CXL vendors drastically want to implement the defered allocation that lazily setup the physical memory to the virtual mmemory, which overlaps the MMAP mechenism.
MMAP + madvise/numabind to certain CXL attached memory may cause migration efforts. Once you dirty write the pages, the transaction is currently not yet introduced in the CXL protocol. The process takes pains to implement the mechesim correctly. Instead, we can do something like TPP or CXLSwap, making everything transparent to applications. Or, we can make 3D memory and extend computability in CXL controller to decide where to put the data and maintain the transaction under the physical memory.
MMAP is originally designed for a fast track memory together with a slower track disk like HDDs. Say you are loading graph edges from a large HDD backed pool. The frequently accessed part will be softwarely defined as a stream pool for cold/hot data management. Here MMAP can both leverage the OS page cache semantic transparently, but it's not case with more and faster endpoints. With more complexity of topology of CXL NUMA devices, we could handle fewer error at a time and serve more for the speed of main bus. Thus, we don't stop for page fault and requires those be handled in endpoints side.
Thus we still need SMDK such management layer to make jemalloc+libnuma+CXLSwap for CXL.mem. For interface with CXL.cache devices, I think defer allocation and managing everything through virtual memory would be fine. Thus we don't need programming models like CUDA; rather, we can static analysis through MLIR to do good data movement hint to every CXL controller's MMU and TLB. We could leverage CXL.cache cacheline state to treat as streaming buffer so that every possible endpoints read for and then do updates by next write.
Chase edges across servers Various neighborhood queries
Extended buffer pool Semantic cache Temporary-data store
Avoiding strictly serializing the roud tip RPC
Avoiding strictly serialized full round trip RPC overheads
Index lookups Predicates on MVs
Sproc (Stored Procedure) and eRPC abstractions
An application specifies a sproc by calling the Register function and executes it by calling the Execute function.
CompuCache uses eRPC, a user-space RPC library that runs on DPDK or RDMA.
Sproc code is a parameter to Register and is broadcast to all CompuCache servers.
DPDK avoids OS kernel overhead by executing the networking in user space.
On each server, the code is compiled locally as a dynamic library and loaded into the runtime of the server.
RDMA offers the further benefit of offloading the CPU by running most of the networking protocol on the network interface card (NIC).
A CompuCache server might not have all the data needed to execute a sproc, which requires coordination with other CompuCache servers
To leverage the full bandwidth of high-speed networks, CompuCache batches small operations in a single network transfer. This includes batching small I/O requests and responses and all sproc execution requests.
We have server-side Pointer Chasing by LocalTranslator Data Structure provides Translate.
An sproc invokes this function to map virtual cache addresses into physical server locations.