Based on kernel version 4.16.1. Page generated on 2018-04-09 11:53 EST.
1 x86 Topology 2 ============ 3 4 This documents and clarifies the main aspects of x86 topology modelling and 5 representation in the kernel. Update/change when doing changes to the 6 respective code. 7 8 The architecture-agnostic topology definitions are in 9 Documentation/cputopology.txt. This file holds x86-specific 10 differences/specialities which must not necessarily apply to the generic 11 definitions. Thus, the way to read up on Linux topology on x86 is to start 12 with the generic one and look at this one in parallel for the x86 specifics. 13 14 Needless to say, code should use the generic functions - this file is *only* 15 here to *document* the inner workings of x86 topology. 16 17 Started by Thomas Gleixner <tglx@linutronix.de> and Borislav Petkov <bp@alien8.de>. 18 19 The main aim of the topology facilities is to present adequate interfaces to 20 code which needs to know/query/use the structure of the running system wrt 21 threads, cores, packages, etc. 22 23 The kernel does not care about the concept of physical sockets because a 24 socket has no relevance to software. It's an electromechanical component. In 25 the past a socket always contained a single package (see below), but with the 26 advent of Multi Chip Modules (MCM) a socket can hold more than one package. So 27 there might be still references to sockets in the code, but they are of 28 historical nature and should be cleaned up. 29 30 The topology of a system is described in the units of: 31 32 - packages 33 - cores 34 - threads 35 36 * Package: 37 38 Packages contain a number of cores plus shared resources, e.g. DRAM 39 controller, shared caches etc. 40 41 AMD nomenclature for package is 'Node'. 42 43 Package-related topology information in the kernel: 44 45 - cpuinfo_x86.x86_max_cores: 46 47 The number of cores in a package. This information is retrieved via CPUID. 48 49 - cpuinfo_x86.phys_proc_id: 50 51 The physical ID of the package. This information is retrieved via CPUID 52 and deduced from the APIC IDs of the cores in the package. 53 54 - cpuinfo_x86.logical_id: 55 56 The logical ID of the package. As we do not trust BIOSes to enumerate the 57 packages in a consistent way, we introduced the concept of logical package 58 ID so we can sanely calculate the number of maximum possible packages in 59 the system and have the packages enumerated linearly. 60 61 - topology_max_packages(): 62 63 The maximum possible number of packages in the system. Helpful for per 64 package facilities to preallocate per package information. 65 66 - cpu_llc_id: 67 68 A per-CPU variable containing: 69 - On Intel, the first APIC ID of the list of CPUs sharing the Last Level 70 Cache 71 72 - On AMD, the Node ID or Core Complex ID containing the Last Level 73 Cache. In general, it is a number identifying an LLC uniquely on the 74 system. 75 76 * Cores: 77 78 A core consists of 1 or more threads. It does not matter whether the threads 79 are SMT- or CMT-type threads. 80 81 AMDs nomenclature for a CMT core is "Compute Unit". The kernel always uses 82 "core". 83 84 Core-related topology information in the kernel: 85 86 - smp_num_siblings: 87 88 The number of threads in a core. The number of threads in a package can be 89 calculated by: 90 91 threads_per_package = cpuinfo_x86.x86_max_cores * smp_num_siblings 92 93 94 * Threads: 95 96 A thread is a single scheduling unit. It's the equivalent to a logical Linux 97 CPU. 98 99 AMDs nomenclature for CMT threads is "Compute Unit Core". The kernel always 100 uses "thread". 101 102 Thread-related topology information in the kernel: 103 104 - topology_core_cpumask(): 105 106 The cpumask contains all online threads in the package to which a thread 107 belongs. 108 109 The number of online threads is also printed in /proc/cpuinfo "siblings." 110 111 - topology_sibling_cpumask(): 112 113 The cpumask contains all online threads in the core to which a thread 114 belongs. 115 116 - topology_logical_package_id(): 117 118 The logical package ID to which a thread belongs. 119 120 - topology_physical_package_id(): 121 122 The physical package ID to which a thread belongs. 123 124 - topology_core_id(); 125 126 The ID of the core to which a thread belongs. It is also printed in /proc/cpuinfo 127 "core_id." 128 129 130 131 System topology examples 132 133 Note: 134 135 The alternative Linux CPU enumeration depends on how the BIOS enumerates the 136 threads. Many BIOSes enumerate all threads 0 first and then all threads 1. 137 That has the "advantage" that the logical Linux CPU numbers of threads 0 stay 138 the same whether threads are enabled or not. That's merely an implementation 139 detail and has no practical impact. 140 141 1) Single Package, Single Core 142 143 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 144 145 2) Single Package, Dual Core 146 147 a) One thread per core 148 149 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 150 -> [core 1] -> [thread 0] -> Linux CPU 1 151 152 b) Two threads per core 153 154 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 155 -> [thread 1] -> Linux CPU 1 156 -> [core 1] -> [thread 0] -> Linux CPU 2 157 -> [thread 1] -> Linux CPU 3 158 159 Alternative enumeration: 160 161 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 162 -> [thread 1] -> Linux CPU 2 163 -> [core 1] -> [thread 0] -> Linux CPU 1 164 -> [thread 1] -> Linux CPU 3 165 166 AMD nomenclature for CMT systems: 167 168 [node 0] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 0 169 -> [Compute Unit Core 1] -> Linux CPU 1 170 -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 2 171 -> [Compute Unit Core 1] -> Linux CPU 3 172 173 4) Dual Package, Dual Core 174 175 a) One thread per core 176 177 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 178 -> [core 1] -> [thread 0] -> Linux CPU 1 179 180 [package 1] -> [core 0] -> [thread 0] -> Linux CPU 2 181 -> [core 1] -> [thread 0] -> Linux CPU 3 182 183 b) Two threads per core 184 185 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 186 -> [thread 1] -> Linux CPU 1 187 -> [core 1] -> [thread 0] -> Linux CPU 2 188 -> [thread 1] -> Linux CPU 3 189 190 [package 1] -> [core 0] -> [thread 0] -> Linux CPU 4 191 -> [thread 1] -> Linux CPU 5 192 -> [core 1] -> [thread 0] -> Linux CPU 6 193 -> [thread 1] -> Linux CPU 7 194 195 Alternative enumeration: 196 197 [package 0] -> [core 0] -> [thread 0] -> Linux CPU 0 198 -> [thread 1] -> Linux CPU 4 199 -> [core 1] -> [thread 0] -> Linux CPU 1 200 -> [thread 1] -> Linux CPU 5 201 202 [package 1] -> [core 0] -> [thread 0] -> Linux CPU 2 203 -> [thread 1] -> Linux CPU 6 204 -> [core 1] -> [thread 0] -> Linux CPU 3 205 -> [thread 1] -> Linux CPU 7 206 207 AMD nomenclature for CMT systems: 208 209 [node 0] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 0 210 -> [Compute Unit Core 1] -> Linux CPU 1 211 -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 2 212 -> [Compute Unit Core 1] -> Linux CPU 3 213 214 [node 1] -> [Compute Unit 0] -> [Compute Unit Core 0] -> Linux CPU 4 215 -> [Compute Unit Core 1] -> Linux CPU 5 216 -> [Compute Unit 1] -> [Compute Unit Core 0] -> Linux CPU 6 217 -> [Compute Unit Core 1] -> Linux CPU 7