Cha and core discovery on intel chips and generating optimized thread binding

In modern multi-core architectures with distributed directory-based cache coherence, each memory address is overseen by a distributed directory unit, known as a Caching/Home Agent (CHA), that monitors cache line state and location. Neither the CHA nor core locations in a processor are directly exposed to the programmer. In this work, we firstly analyze and compare the methodologies for uncovering both the CHA and core topology of Intel Xeon Scalable processors, as well as the methods to reveal the mapping of memory addresses to CHAs. Leveraging the topology and the address mapping information, we investigate the impact of spatial proximity between communicating cores and CHAs on application performance, and propose a thread mapping heuristic that assigns threads to cores by considering cache coherence traffic. We expect our heuristic to achieve significant performance gains on applications with high amount of on-chip cache coherence traffic due to high percentage of shared written data. We evaluated our heuristic on applications that exhibit high amount of on-chip communication traffic. The heuristic achieves up to 5.6% speedup over compact placement on merge-based SpMV application, up to 8% with an average of around 4.4% on Barnes application, around 25% for Fluidanimate application to simulate 60 frame per second, and lastly approximately 6% for LU across different matrices. We also prove the improved performance is in fact related to reduced on-chip traffic on the mesh.