Category: Study 📕

Very interesting!

When I reviewed the paper in the past, I was surprised that the recently proposed plans like JIT, MLIR, and eBPF could be a great fit for the legacy tools like record and replay and security live patching or kernel modeling.

Reference

1. https://www.usenix.org/legacy/event/osdi08/tech/full_papers/guo/guo.pdf
2. https://nimrodpar.github.io/assets/publications/rr.pdf
3. https://iacoma.cs.uiuc.edu/iacoma-papers/hpca18.pdf

The auto-flushing-persistently from GPU to PMEM without serializing the GPU memory makes it the greatest in snapshotting or swapping. When I was rewriting QuEST in ASC 21-22, we found that Tsinghua University Zhangcheng and NUDT is doing swapping memory back and forth to make it possible to emulate more QuBits. This arch will be a best fit for that.

Their story is a recommendation system that the embedding table could not fit in a single card and disaggregate the GPU memory illustration of the embedding table to other CXL2.0 devices.

So still taking the memory as an outer device but not integrating the fault tolerance into the protocol level to make sure the single node fault tolerance. Still, it's a pure hardware stack implemented software check-pointing idea. Way brute forcing and nontransferable to other workloads. The flag and MLP unique features augmented is the basic idea of write log we were researching on the persistent memory file systems.

Their emulation platform is ... I'm shocked... The GPU emulation is Vortex. They don't actually need both the GPU and CPU to support the CXL controller interface. They just want to make cacheline shared in the CXl pool between Intel Optane and GPU. However, I think this kind of emulation is not accurate enough for CPU internal cacheline mechanism.

Reference

1. https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10018233&tag=1
2. https://arxiv.org/pdf/2301.07492.pdf

This document will be updated in wiki

1. CCD interconnect Metrics
   1. Since the interconnect of the CCDs is EMIB, and the cache coherency is still directory routing from LLC to HBM and Memory, the PMU has operands DRd/CRd and abstraction of TOR and IA to record the metrics.
   2. All CHA Uncore PMU starts with UNC_CHA_TOR_INSERTS, those can be labeled as CHA tile 0x2000-0x2180 for a 23-core machine, which maps to each tile's 2/3/4/5 are control registers and 8/9/a/b are result registers. It's easy to get cha id mapping to os id through unc memory bandwidth write.
   3. Every local CCD mesh has 3 UPIs, 3 memory port and 1 PCIe &CXL port. There are three types of traffic IIO/PCIe&CXL/UPI. From 0x3000-0x30b0, Only 1 has value(because there's no PCIe device attached) on my machine for PCIe1-4 IIO 1-4, which has 4 control registers, which maps to each tile's 2/3/4/5 are control register and 8/9/a/b are result registers. From 0x3400-0x34b0, there are only 6 values one my machine, which I think should be UPI and iMC ports. All of those only have 2 control registers, 2/3 are control registers and 8/9 are result registers.
2. CXL Traffic Metrics
   1. I think the current PMU is not fully renamed to the CXL.cache but embedded in the core's metrics, like hitm, miss snoop_none, etc.
   2. OCR.READS_TO_CORE.L3_MISS_LOCAL_SOCKET. This metric will count all the code read and RFO that miss the L3 but request to local CXL Type 2 or private DRAM. Remote requests will count toward the remote counterpart metric.
3. Some guesses
   1. HBM2E serves as a non-inclusive directory-based directly mapped L4 cache(manifested as HBM M2M(mesh to memory) blocks and EDC/HBM channels) like persistent memory using the extra tag for caching. It can cache some of the CXL.cache requests if LLC each CCD is not big enough.
   2. AMD supports CXL2.0's RAS and gpf, while Intel gives it up because 1. gpf is useless, 2. it's the outer fabric manager's work, and intel puts the efforts into vendors.

Reference

1. https://zhuanlan.zhihu.com/p/598702329
2. https://perfmon-events.intel.com/index.html?pltfrm=ahybrid.html&evnt=OCR.DEMAND_DATA_RD.L3_MISS

All the variable is SimObj in Gem5. For instance, the SignaturePath Prefetcher is an instance of QueuedPrefetcher, The parameter to instantiate the object is passed with

struct SignaturePathPrefetcherParams
    : public QueuedPrefetcherParams
{
    gem5::prefetch::SignaturePath * create() const;
    double lookahead_confidence_threshold;
    uint8_t num_counter_bits;
    unsigned pattern_table_assoc;
    uint64_t pattern_table_entries;
    gem5::BaseIndexingPolicy * pattern_table_indexing_policy;
    gem5::replacement_policy::Base * pattern_table_replacement_policy;
    double prefetch_confidence_threshold;
    uint16_t signature_bits;
    uint8_t signature_shift;
    unsigned signature_table_assoc;
    uint64_t signature_table_entries;
    gem5::BaseIndexingPolicy * signature_table_indexing_policy;
    gem5::replacement_policy::Base * signature_table_replacement_policy;
    unsigned strides_per_pattern_entry;
};

Because there are actually 3 entries defined in SPPPrefetcher, the version 2 of SPP add the Global History Register Table, every time you accessed the prefetcher, say grabbing information from L2 access info, is adding an entry in these three tables. And the timing of fetching is during the pipeline fetching the data.

The associativity of the table inside SPPPrefetcher is always 1. So the Base IndexPolicy is too tooo toooo overly enculpsion for this prefetcher.

The BaseIndexPolicy is every time you access the entry, the AssociativeSet will initialize a set resides the vector you access, say a TaggedEntry that uses the address as a tag, it will also initialize the ReplcamentPolicy basically requiring the entry vector and pattern vector. every time find a victim will call the replacement policy and call LRU to evict the victim. I will do it using a lambda function over vector view.