dankwiki

SMP on x86: Difference between revisions

← Older edit

Dank (talk | contribs)

2 user groups
11,302 edits
No edit summary
 
(31 intermediate revisions by the same user not shown)
Line 1: Line 1:
The primary specification for multiprocessor [[x86]]-based setups is the [http://www.intel.com/design/pentium/datashts/242016.HTM Intel MultiProcessor Specification] (last updated, AFAIK, to revision-006 on 1995-05-15).
The primary specification for multiprocessor [[x86]]-based setups is the [http://www.intel.com/design/pentium/datashts/242016.HTM Intel MultiProcessor Specification] (last updated, AFAIK, to revision-006 on 1995-05-15).


* [[ACPI]]
==SMT==
** The MADT table can supply multiprocessor configuration
* HyperThreading (Intel SMT) requires CPU, BIOS and OS support. Introduced on the P4.
* MP Table
** Found on (all) i7's, [http://www.intel.com/technology/atom/microarchitecture.htm some Atoms], and some P4's and Core2Duo's (especially those with Xeon branding).
* Interactions with [[cpuid]]
** Pentium-M and Celerons usually lack SMT.
* HyperThreading requires CPU, BIOS and OS support. It's found on some P4's and Core2Duo's, especially from the Xeon line.
* Unduplicated resources are either split or shared between logical cores:
** Shared: reservation stations, data caches
** Split: reorder buffers, load/store buffers
** Duplicated: registers
* No programmable priority control exported, no implicit priorities defined


==libvirt==
==Intel MP IDs==
* <tt>virsh(1)</tt>'s <tt>nodeinfo</tt> command can be pretty useful:
* Each logical processor is assigned a unique (but not necessarily sequential) 8-bit identifier at boot, the APIC ID
<pre>[wopr](0) $ virsh nodeinfo
** The initial APIC ID can be retrieved via [[cpuid]], as can packaging data:
CPU model:           x86_64
*** "logical cores per package" (CPUID.1.EBX[23:16]): Maximum number of logical processors in a physical package, as manufactured
CPU(s):             16
*** "cores per package" (CPUID.4.EAX[31:26] + 1): Maximum number of physical processors (cores) in a physical package, as manufactured
CPU frequency:       1600 MHz
*** "logical processors sharing a cache" (CPUID.4.EAX[25:14] + 1): Maximum number of logical processors in a physical package sharing a given cachelevel
CPU socket(s):       4
*** Intel MP only addresses homogeneous setups, so these three values are (as of October 2009) equivalent for all processors
Core(s) per socket: 4
*** These last two require leaf level 4 [[cpuid]] support; if it is not provided, the package is a unicore
Thread(s) per core: 1
* APIC ID is formed of SMT_ID|CORE_ID|PACKAGE_ID, having widths defined by the packaging data:
NUMA cell(s):       1
** SMT_ID is 0 bits on a non-HyperThreaded processor.
Memory size:         66113480 kB
** CORE_ID is 0 bits on a unicore package
** All remaining bits are devoted to PACKAGE_ID
 
==Interrupts==
* IO-APIC routes hardware interrupts to various CPUs ([http://www.mjmwired.net/kernel/Documentation/x86/i386/IO-APIC.txt Linux's IO-APIC.txt])
<pre>[recombinator](0) $ cat /proc/interrupts
          CPU0      CPU1     
  0:        491          0  IO-APIC-edge      timer
  8:        87          0  IO-APIC-edge      rtc0
  9:          0          0  IO-APIC-fasteoi  acpi
16:    609652          0  IO-APIC-fasteoi  uhci_hcd:usb1, heci
17:          0          0  IO-APIC-fasteoi  pata_marvell
18:      33141          0  IO-APIC-fasteoi  sata_promise, uhci_hcd:usb5, ehci_hcd:usb6
19:          0          0  IO-APIC-fasteoi  uhci_hcd:usb4
21:  15985105          0  IO-APIC-fasteoi  uhci_hcd:usb2, ath
23:    8730160          0  IO-APIC-fasteoi  uhci_hcd:usb3, ehci_hcd:usb7
29:    1855556          0  PCI-MSI-edge      i915
30:    1109316          0  PCI-MSI-edge      ahci
31:      66952          0  PCI-MSI-edge      e1000
NMI:          0          0  Non-maskable interrupts
LOC:  14417376  16273096  Local timer interrupts
SPU:          0          0  Spurious interrupts
RES:    150950    182106  Rescheduling interrupts
CAL:        278        611  Function call interrupts
TLB:      33349      53705  TLB shootdowns
TRM:          0          0  Thermal event interrupts
THR:          0          0  Threshold APIC interrupts
ERR:          0
MIS:          0
[recombinator](0) $ </pre>
==Discovery==
* [[ACPI]]'s MADT table can supply multiprocessor configuration, and is usually used if present
* MP Table. The FreeBSD program <tt>mptable(1)</tt> can dump this:
<pre>[bryhlath](0) $ sudo mptable
 
===============================================================================
 
MPTable
 
-------------------------------------------------------------------------------
 
MP Floating Pointer Structure:
 
  location: BIOS
  physical address: 0x000fbd10
  signature: '_MP_'
  length: 16 bytes
  version: 1.4
  checksum: 0xc6
  mode: Virtual Wire
 
-------------------------------------------------------------------------------
 
MP Config Table Header:
 
  physical address: 0x000fbb10
  signature: 'PCMP'
  base table length: 508
  version: 1.4
  checksum: 0x84
  OEM ID: 'QEMUCPU '
  Product ID: '0.1         '
  OEM table pointer: 0x00000000
  OEM table size: 0
  entry count: 34
  local APIC address: 0xfee00000
  extended table length: 0
  extended table checksum: 0
 
-------------------------------------------------------------------------------
 
MP Config Base Table Entries:


[wopr](0) $ </pre>
--
Processors: APIC ID Version State Family Model Step Flags
0 0x11 BSP, usable 6 0 0 0x0201
1 0x11 AP, unusable 6 0 0 0x0201
2 0x11 AP, unusable 6 0 0 0x0201
3 0x11 AP, unusable 6 0 0 0x0201
4 0x11 AP, unusable 6 0 0 0x0201
5 0x11 AP, unusable 6 0 0 0x0201
6 0x11 AP, unusable 6 0 0 0x0201
7 0x11 AP, unusable 6 0 0 0x0201
8 0x11 AP, unusable 6 0 0 0x0201
9 0x11 AP, unusable 6 0 0 0x0201
10 0x11 AP, unusable 6 0 0 0x0201
11 0x11 AP, unusable 6 0 0 0x0201
12 0x11 AP, unusable 6 0 0 0x0201
13 0x11 AP, unusable 6 0 0 0x0201
14 0x11 AP, unusable 6 0 0 0x0201
15 0x11 AP, unusable 6 0 0 0x0201
--
Bus: Bus ID Type
0 ISA 
--
I/O APICs: APIC ID Version State Address
1 0x11 usable 0xfec00000
--
I/O Ints: Type Polarity    Trigger Bus ID IRQ APIC ID PIN#
INT conforms    conforms     0   0       1   0
INT conforms    conforms     0   1       1   1
INT conforms    conforms     0   2       1   2
INT conforms    conforms     0   3       1   3
INT conforms    conforms     0   4       1   4
INT conforms    conforms     0   5       1   5
INT conforms    conforms     0   6       1   6
INT conforms    conforms     0   7       1   7
INT conforms    conforms     0   8       1   8
INT conforms    conforms     0   9       1   9
INT conforms    conforms     0   10       1   10
INT conforms    conforms     0   11       1   11
INT conforms    conforms     0   12       1   12
INT conforms    conforms     0   13       1   13
INT conforms    conforms     0   14       1   14
INT conforms    conforms     0   15       1   15


<pre>[recombinator](0) $ sudo virsh nodeinfo
===============================================================================
CPU model:          x86_64
CPU(s):              2
CPU frequency:      1596 MHz
CPU socket(s):      1
Core(s) per socket:  2
Thread(s) per core:  1
NUMA cell(s):        1
Memory size:        3908568 kB


[recombinator](0) $ </pre>
[bryhlath](0) $ </pre>


==/proc/cpuinfo==
==/proc/cpuinfo==
Line 44: Line 152:
I've removed all output from the following examples, save that related to SMP identification.
I've removed all output from the following examples, save that related to SMP identification.


*EMT64 Xeon, no HyperThreading support, 4 cores per socket, 4 sockets: 16 total execution units (2.6.26)
*EMT64 Xeon, no HyperThreading support, 4 cores per socket, 4 sockets: 16 total execution units (2.6.26) (Dell R900)
<pre>[wopr](0) $ cat /proc/cpuinfo | egrep ^proc\|^model\ \|^phys\|^sib\|^core\|^cpu\ c\|^ap\|^init
<pre>[wopr](0) $ cat /proc/cpuinfo | egrep ^proc\|^model\ \|^phys\|^sib\|^core\|^cpu\ c\|^ap\|^init
processor : 0
processor : 0
Line 190: Line 298:
initial apicid : 27
initial apicid : 27
[wopr](0) $ </pre>
[wopr](0) $ </pre>
*Xeon E5520, HyperThreading enabled, 4 cores per socket, 2 sockets: 16 total execution units (Dell R710)
<pre>[dumbledore](0) $ egrep '^(proc|phys|sib|acpi|model )|core' /proc/cpuinfo
processor : 0
model name : Intel(R) Xeon(R) CPU          E5520  @ 2.27GHz
physical id : 1
siblings : 8
core id : 0
cpu cores : 4
processor : 1
model name : Intel(R) Xeon(R) CPU          E5520  @ 2.27GHz
physical id : 0
siblings : 8
core id : 0
cpu cores : 4
processor : 2
model name : Intel(R) Xeon(R) CPU          E5520  @ 2.27GHz
physical id : 1
siblings : 8
core id : 1
cpu cores : 4
processor : 3
model name : Intel(R) Xeon(R) CPU          E5520  @ 2.27GHz
physical id : 0
siblings : 8
core id : 1
cpu cores : 4
processor : 4
model name : Intel(R) Xeon(R) CPU          E5520  @ 2.27GHz
physical id : 1
siblings : 8
core id : 2
cpu cores : 4
processor : 5
model name : Intel(R) Xeon(R) CPU          E5520  @ 2.27GHz
physical id : 0
siblings : 8
core id : 2
cpu cores : 4
processor : 6
model name : Intel(R) Xeon(R) CPU          E5520  @ 2.27GHz
physical id : 1
siblings : 8
core id : 3
cpu cores : 4
processor : 7
model name : Intel(R) Xeon(R) CPU          E5520  @ 2.27GHz
physical id : 0
siblings : 8
core id : 3
cpu cores : 4
processor : 8
model name : Intel(R) Xeon(R) CPU          E5520  @ 2.27GHz
physical id : 1
siblings : 8
core id : 0
cpu cores : 4
processor : 9
model name : Intel(R) Xeon(R) CPU          E5520  @ 2.27GHz
physical id : 0
siblings : 8
core id : 0
cpu cores : 4
processor : 10
model name : Intel(R) Xeon(R) CPU          E5520  @ 2.27GHz
physical id : 1
siblings : 8
core id : 1
cpu cores : 4
processor : 11
model name : Intel(R) Xeon(R) CPU          E5520  @ 2.27GHz
physical id : 0
siblings : 8
core id : 1
cpu cores : 4
processor : 12
model name : Intel(R) Xeon(R) CPU          E5520  @ 2.27GHz
physical id : 1
siblings : 8
core id : 2
cpu cores : 4
processor : 13
model name : Intel(R) Xeon(R) CPU          E5520  @ 2.27GHz
physical id : 0
siblings : 8
core id : 2
cpu cores : 4
processor : 14
model name : Intel(R) Xeon(R) CPU          E5520  @ 2.27GHz
physical id : 1
siblings : 8
core id : 3
cpu cores : 4
processor : 15
model name : Intel(R) Xeon(R) CPU          E5520  @ 2.27GHz
physical id : 0
siblings : 8
core id : 3
cpu cores : 4
[dumbledore](0) $ </pre>
*Core 2 Duo, no HyperThreading support, 2 cores per socket, 1 socket: 2 total execution units (2.6.26)
*Core 2 Duo, no HyperThreading support, 2 cores per socket, 1 socket: 2 total execution units (2.6.26)
<pre>[recombinator](0) $ cat /proc/cpuinfo  
<pre>[recombinator](0) $ cat /proc/cpuinfo  
Line 259: Line 466:
* <tt>machdep.hyperthreading.allowed</tt>
* <tt>machdep.hyperthreading.allowed</tt>
* <tt>kern.threads.virtual_cpu</tt>
* <tt>kern.threads.virtual_cpu</tt>
==libvirt==
* <tt>virsh(1)</tt>'s <tt>nodeinfo</tt> command can be pretty useful:
<pre>[wopr](0) $ virsh nodeinfo
CPU model:          x86_64
CPU(s):              16
CPU frequency:      1600 MHz
CPU socket(s):      4
Core(s) per socket:  4
Thread(s) per core:  1
NUMA cell(s):        1
Memory size:        66113480 kB
[wopr](0) $ </pre>
<pre>[recombinator](0) $ sudo virsh nodeinfo
CPU model:          x86_64
CPU(s):              2
CPU frequency:      1596 MHz
CPU socket(s):      1
Core(s) per socket:  2
Thread(s) per core:  1
NUMA cell(s):        1
Memory size:        3908568 kB
[recombinator](0) $ </pre>


==See also==
==See also==
* [http://osdir.com/ml/linux.ports.x86-64.general/2005-10/msg00022.html This LKML thread] provides much information
* [http://osdir.com/ml/linux.ports.x86-64.general/2005-10/msg00022.html This LKML thread] provides much information
* The [[cpuid]] instruction can interrogate each processing unit
* [[Cpuset|CPUsets]]
* "[http://software.intel.com/en-us/articles/multi-core-detect/ Detecting Multicore Processors]", Intel Software Network
[[Category: x86]]
[[CATEGORY: Hardware]]
Retrieved from "https://nick-black.com/dankwiki/index.php?title=SMP_on_x86"