Based on kernel version 4.16.1. Page generated on 2018-04-09 11:53 EST.
1 Kernel stacks on x86-64 bit 2 --------------------------- 3 4 Most of the text from Keith Owens, hacked by AK 5 6 x86_64 page size (PAGE_SIZE) is 4K. 7 8 Like all other architectures, x86_64 has a kernel stack for every 9 active thread. These thread stacks are THREAD_SIZE (2*PAGE_SIZE) big. 10 These stacks contain useful data as long as a thread is alive or a 11 zombie. While the thread is in user space the kernel stack is empty 12 except for the thread_info structure at the bottom. 13 14 In addition to the per thread stacks, there are specialized stacks 15 associated with each CPU. These stacks are only used while the kernel 16 is in control on that CPU; when a CPU returns to user space the 17 specialized stacks contain no useful data. The main CPU stacks are: 18 19 * Interrupt stack. IRQ_STACK_SIZE 20 21 Used for external hardware interrupts. If this is the first external 22 hardware interrupt (i.e. not a nested hardware interrupt) then the 23 kernel switches from the current task to the interrupt stack. Like 24 the split thread and interrupt stacks on i386, this gives more room 25 for kernel interrupt processing without having to increase the size 26 of every per thread stack. 27 28 The interrupt stack is also used when processing a softirq. 29 30 Switching to the kernel interrupt stack is done by software based on a 31 per CPU interrupt nest counter. This is needed because x86-64 "IST" 32 hardware stacks cannot nest without races. 33 34 x86_64 also has a feature which is not available on i386, the ability 35 to automatically switch to a new stack for designated events such as 36 double fault or NMI, which makes it easier to handle these unusual 37 events on x86_64. This feature is called the Interrupt Stack Table 38 (IST). There can be up to 7 IST entries per CPU. The IST code is an 39 index into the Task State Segment (TSS). The IST entries in the TSS 40 point to dedicated stacks; each stack can be a different size. 41 42 An IST is selected by a non-zero value in the IST field of an 43 interrupt-gate descriptor. When an interrupt occurs and the hardware 44 loads such a descriptor, the hardware automatically sets the new stack 45 pointer based on the IST value, then invokes the interrupt handler. If 46 the interrupt came from user mode, then the interrupt handler prologue 47 will switch back to the per-thread stack. If software wants to allow 48 nested IST interrupts then the handler must adjust the IST values on 49 entry to and exit from the interrupt handler. (This is occasionally 50 done, e.g. for debug exceptions.) 51 52 Events with different IST codes (i.e. with different stacks) can be 53 nested. For example, a debug interrupt can safely be interrupted by an 54 NMI. arch/x86_64/kernel/entry.S::paranoidentry adjusts the stack 55 pointers on entry to and exit from all IST events, in theory allowing 56 IST events with the same code to be nested. However in most cases, the 57 stack size allocated to an IST assumes no nesting for the same code. 58 If that assumption is ever broken then the stacks will become corrupt. 59 60 The currently assigned IST stacks are :- 61 62 * DOUBLEFAULT_STACK. EXCEPTION_STKSZ (PAGE_SIZE). 63 64 Used for interrupt 8 - Double Fault Exception (#DF). 65 66 Invoked when handling one exception causes another exception. Happens 67 when the kernel is very confused (e.g. kernel stack pointer corrupt). 68 Using a separate stack allows the kernel to recover from it well enough 69 in many cases to still output an oops. 70 71 * NMI_STACK. EXCEPTION_STKSZ (PAGE_SIZE). 72 73 Used for non-maskable interrupts (NMI). 74 75 NMI can be delivered at any time, including when the kernel is in the 76 middle of switching stacks. Using IST for NMI events avoids making 77 assumptions about the previous state of the kernel stack. 78 79 * DEBUG_STACK. DEBUG_STKSZ 80 81 Used for hardware debug interrupts (interrupt 1) and for software 82 debug interrupts (INT3). 83 84 When debugging a kernel, debug interrupts (both hardware and 85 software) can occur at any time. Using IST for these interrupts 86 avoids making assumptions about the previous state of the kernel 87 stack. 88 89 * MCE_STACK. EXCEPTION_STKSZ (PAGE_SIZE). 90 91 Used for interrupt 18 - Machine Check Exception (#MC). 92 93 MCE can be delivered at any time, including when the kernel is in the 94 middle of switching stacks. Using IST for MCE events avoids making 95 assumptions about the previous state of the kernel stack. 96 97 For more details see the Intel IA32 or AMD AMD64 architecture manuals. 98 99 100 Printing backtraces on x86 101 -------------------------- 102 103 The question about the '?' preceding function names in an x86 stacktrace 104 keeps popping up, here's an indepth explanation. It helps if the reader 105 stares at print_context_stack() and the whole machinery in and around 106 arch/x86/kernel/dumpstack.c. 107 108 Adapted from Ingo's mail, Message-ID: <20150521101614.GA10889@gmail.com>: 109 110 We always scan the full kernel stack for return addresses stored on 111 the kernel stack(s) [*], from stack top to stack bottom, and print out 112 anything that 'looks like' a kernel text address. 113 114 If it fits into the frame pointer chain, we print it without a question 115 mark, knowing that it's part of the real backtrace. 116 117 If the address does not fit into our expected frame pointer chain we 118 still print it, but we print a '?'. It can mean two things: 119 120 - either the address is not part of the call chain: it's just stale 121 values on the kernel stack, from earlier function calls. This is 122 the common case. 123 124 - or it is part of the call chain, but the frame pointer was not set 125 up properly within the function, so we don't recognize it. 126 127 This way we will always print out the real call chain (plus a few more 128 entries), regardless of whether the frame pointer was set up correctly 129 or not - but in most cases we'll get the call chain right as well. The 130 entries printed are strictly in stack order, so you can deduce more 131 information from that as well. 132 133 The most important property of this method is that we _never_ lose 134 information: we always strive to print _all_ addresses on the stack(s) 135 that look like kernel text addresses, so if debug information is wrong, 136 we still print out the real call chain as well - just with more question 137 marks than ideal. 138 139 [*] For things like IRQ and IST stacks, we also scan those stacks, in 140 the right order, and try to cross from one stack into another 141 reconstructing the call chain. This works most of the time.