About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Documentation / gdb-kernel-debugging.txt




Custom Search

Based on kernel version 4.8. Page generated on 2016-10-06 23:16 EST.

1	Debugging kernel and modules via gdb
2	====================================
3	
4	The kernel debugger kgdb, hypervisors like QEMU or JTAG-based hardware
5	interfaces allow to debug the Linux kernel and its modules during runtime
6	using gdb. Gdb comes with a powerful scripting interface for python. The
7	kernel provides a collection of helper scripts that can simplify typical
8	kernel debugging steps. This is a short tutorial about how to enable and use
9	them. It focuses on QEMU/KVM virtual machines as target, but the examples can
10	be transferred to the other gdb stubs as well.
11	
12	
13	Requirements
14	------------
15	
16	 o gdb 7.2+ (recommended: 7.4+) with python support enabled (typically true
17	   for distributions)
18	
19	
20	Setup
21	-----
22	
23	 o Create a virtual Linux machine for QEMU/KVM (see www.linux-kvm.org and
24	   www.qemu.org for more details). For cross-development,
25	   http://landley.net/aboriginal/bin keeps a pool of machine images and
26	   toolchains that can be helpful to start from.
27	
28	 o Build the kernel with CONFIG_GDB_SCRIPTS enabled, but leave
29	   CONFIG_DEBUG_INFO_REDUCED off. If your architecture supports
30	   CONFIG_FRAME_POINTER, keep it enabled.
31	
32	 o Install that kernel on the guest.
33	
34	   Alternatively, QEMU allows to boot the kernel directly using -kernel,
35	   -append, -initrd command line switches. This is generally only useful if
36	   you do not depend on modules. See QEMU documentation for more details on
37	   this mode.
38	
39	 o Enable the gdb stub of QEMU/KVM, either
40	    - at VM startup time by appending "-s" to the QEMU command line
41	   or
42	    - during runtime by issuing "gdbserver" from the QEMU monitor
43	      console
44	
45	 o cd /path/to/linux-build
46	
47	 o Start gdb: gdb vmlinux
48	
49	   Note: Some distros may restrict auto-loading of gdb scripts to known safe
50	   directories. In case gdb reports to refuse loading vmlinux-gdb.py, add
51	
52	    add-auto-load-safe-path /path/to/linux-build
53	
54	   to ~/.gdbinit. See gdb help for more details.
55	
56	 o Attach to the booted guest:
57	    (gdb) target remote :1234
58	
59	
60	Examples of using the Linux-provided gdb helpers
61	------------------------------------------------
62	
63	 o Load module (and main kernel) symbols:
64	    (gdb) lx-symbols
65	    loading vmlinux
66	    scanning for modules in /home/user/linux/build
67	    loading @0xffffffffa0020000: /home/user/linux/build/net/netfilter/xt_tcpudp.ko
68	    loading @0xffffffffa0016000: /home/user/linux/build/net/netfilter/xt_pkttype.ko
69	    loading @0xffffffffa0002000: /home/user/linux/build/net/netfilter/xt_limit.ko
70	    loading @0xffffffffa00ca000: /home/user/linux/build/net/packet/af_packet.ko
71	    loading @0xffffffffa003c000: /home/user/linux/build/fs/fuse/fuse.ko
72	    ...
73	    loading @0xffffffffa0000000: /home/user/linux/build/drivers/ata/ata_generic.ko
74	
75	 o Set a breakpoint on some not yet loaded module function, e.g.:
76	    (gdb) b btrfs_init_sysfs
77	    Function "btrfs_init_sysfs" not defined.
78	    Make breakpoint pending on future shared library load? (y or [n]) y
79	    Breakpoint 1 (btrfs_init_sysfs) pending.
80	
81	 o Continue the target
82	    (gdb) c
83	
84	 o Load the module on the target and watch the symbols being loaded as well as
85	   the breakpoint hit:
86	    loading @0xffffffffa0034000: /home/user/linux/build/lib/libcrc32c.ko
87	    loading @0xffffffffa0050000: /home/user/linux/build/lib/lzo/lzo_compress.ko
88	    loading @0xffffffffa006e000: /home/user/linux/build/lib/zlib_deflate/zlib_deflate.ko
89	    loading @0xffffffffa01b1000: /home/user/linux/build/fs/btrfs/btrfs.ko
90	
91	    Breakpoint 1, btrfs_init_sysfs () at /home/user/linux/fs/btrfs/sysfs.c:36
92	    36              btrfs_kset = kset_create_and_add("btrfs", NULL, fs_kobj);
93	
94	 o Dump the log buffer of the target kernel:
95	    (gdb) lx-dmesg
96	    [     0.000000] Initializing cgroup subsys cpuset
97	    [     0.000000] Initializing cgroup subsys cpu
98	    [     0.000000] Linux version 3.8.0-rc4-dbg+ (...
99	    [     0.000000] Command line: root=/dev/sda2 resume=/dev/sda1 vga=0x314
100	    [     0.000000] e820: BIOS-provided physical RAM map:
101	    [     0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
102	    [     0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
103	    ....
104	
105	 o Examine fields of the current task struct:
106	    (gdb) p $lx_current().pid
107	    $1 = 4998
108	    (gdb) p $lx_current().comm
109	    $2 = "modprobe\000\000\000\000\000\000\000"
110	
111	 o Make use of the per-cpu function for the current or a specified CPU:
112	    (gdb) p $lx_per_cpu("runqueues").nr_running
113	    $3 = 1
114	    (gdb) p $lx_per_cpu("runqueues", 2).nr_running
115	    $4 = 0
116	
117	 o Dig into hrtimers using the container_of helper:
118	    (gdb) set $next = $lx_per_cpu("hrtimer_bases").clock_base[0].active.next
119	    (gdb) p *$container_of($next, "struct hrtimer", "node")
120	    $5 = {
121	      node = {
122	        node = {
123	          __rb_parent_color = 18446612133355256072,
124	          rb_right = 0x0 <irq_stack_union>,
125	          rb_left = 0x0 <irq_stack_union>
126	        },
127	        expires = {
128	          tv64 = 1835268000000
129	        }
130	      },
131	      _softexpires = {
132	        tv64 = 1835268000000
133	      },
134	      function = 0xffffffff81078232 <tick_sched_timer>,
135	      base = 0xffff88003fd0d6f0,
136	      state = 1,
137	      start_pid = 0,
138	      start_site = 0xffffffff81055c1f <hrtimer_start_range_ns+20>,
139	      start_comm = "swapper/2\000\000\000\000\000\000"
140	    }
141	
142	
143	List of commands and functions
144	------------------------------
145	
146	The number of commands and convenience functions may evolve over the time,
147	this is just a snapshot of the initial version:
148	
149	 (gdb) apropos lx
150	 function lx_current -- Return current task
151	 function lx_module -- Find module by name and return the module variable
152	 function lx_per_cpu -- Return per-cpu variable
153	 function lx_task_by_pid -- Find Linux task by PID and return the task_struct variable
154	 function lx_thread_info -- Calculate Linux thread_info from task variable
155	 lx-dmesg -- Print Linux kernel log buffer
156	 lx-lsmod -- List currently loaded modules
157	 lx-symbols -- (Re-)load symbols of Linux kernel and currently loaded modules
158	
159	Detailed help can be obtained via "help <command-name>" for commands and "help
160	function <function-name>" for convenience functions.
Hide Line Numbers
About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Information is copyright its respective author. All material is available from the Linux Kernel Source distributed under a GPL License. This page is provided as a free service by mjmwired.net.