About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Documentation / dynamic-debug-howto.txt




Custom Search

Based on kernel version 3.2. Page generated on 2012-01-05 23:28 EST.

1	
2	Introduction
3	============
4	
5	This document describes how to use the dynamic debug (ddebug) feature.
6	
7	Dynamic debug is designed to allow you to dynamically enable/disable kernel
8	code to obtain additional kernel information. Currently, if
9	CONFIG_DYNAMIC_DEBUG is set, then all pr_debug()/dev_dbg() calls can be
10	dynamically enabled per-callsite.
11	
12	Dynamic debug has even more useful features:
13	
14	 * Simple query language allows turning on and off debugging statements by
15	   matching any combination of:
16	
17	   - source filename
18	   - function name
19	   - line number (including ranges of line numbers)
20	   - module name
21	   - format string
22	
23	 * Provides a debugfs control file: <debugfs>/dynamic_debug/control which can be
24	   read to display the complete list of known debug statements, to help guide you
25	
26	Controlling dynamic debug Behaviour
27	===================================
28	
29	The behaviour of pr_debug()/dev_dbg()s are controlled via writing to a
30	control file in the 'debugfs' filesystem. Thus, you must first mount the debugfs
31	filesystem, in order to make use of this feature. Subsequently, we refer to the
32	control file as: <debugfs>/dynamic_debug/control. For example, if you want to
33	enable printing from source file 'svcsock.c', line 1603 you simply do:
34	
35	nullarbor:~ # echo 'file svcsock.c line 1603 +p' >
36					<debugfs>/dynamic_debug/control
37	
38	If you make a mistake with the syntax, the write will fail thus:
39	
40	nullarbor:~ # echo 'file svcsock.c wtf 1 +p' >
41					<debugfs>/dynamic_debug/control
42	-bash: echo: write error: Invalid argument
43	
44	Viewing Dynamic Debug Behaviour
45	===========================
46	
47	You can view the currently configured behaviour of all the debug statements
48	via:
49	
50	nullarbor:~ # cat <debugfs>/dynamic_debug/control
51	# filename:lineno [module]function flags format
52	/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:323 [svcxprt_rdma]svc_rdma_cleanup - "SVCRDMA Module Removed, deregister RPC RDMA transport\012"
53	/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:341 [svcxprt_rdma]svc_rdma_init - "\011max_inline       : %d\012"
54	/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:340 [svcxprt_rdma]svc_rdma_init - "\011sq_depth         : %d\012"
55	/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svc_rdma.c:338 [svcxprt_rdma]svc_rdma_init - "\011max_requests     : %d\012"
56	...
57	
58	
59	You can also apply standard Unix text manipulation filters to this
60	data, e.g.
61	
62	nullarbor:~ # grep -i rdma <debugfs>/dynamic_debug/control  | wc -l
63	62
64	
65	nullarbor:~ # grep -i tcp <debugfs>/dynamic_debug/control | wc -l
66	42
67	
68	Note in particular that the third column shows the enabled behaviour
69	flags for each debug statement callsite (see below for definitions of the
70	flags).  The default value, no extra behaviour enabled, is "-".  So
71	you can view all the debug statement callsites with any non-default flags:
72	
73	nullarbor:~ # awk '$3 != "-"' <debugfs>/dynamic_debug/control
74	# filename:lineno [module]function flags format
75	/usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svcsock.c:1603 [sunrpc]svc_send p "svc_process: st_sendto returned %d\012"
76	
77	
78	Command Language Reference
79	==========================
80	
81	At the lexical level, a command comprises a sequence of words separated
82	by whitespace characters.  Note that newlines are treated as word
83	separators and do *not* end a command or allow multiple commands to
84	be done together.  So these are all equivalent:
85	
86	nullarbor:~ # echo -c 'file svcsock.c line 1603 +p' >
87					<debugfs>/dynamic_debug/control
88	nullarbor:~ # echo -c '  file   svcsock.c     line  1603 +p  ' >
89					<debugfs>/dynamic_debug/control
90	nullarbor:~ # echo -c 'file svcsock.c\nline 1603 +p' >
91					<debugfs>/dynamic_debug/control
92	nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' >
93					<debugfs>/dynamic_debug/control
94	
95	Commands are bounded by a write() system call.  If you want to do
96	multiple commands you need to do a separate "echo" for each, like:
97	
98	nullarbor:~ # echo 'file svcsock.c line 1603 +p' > /proc/dprintk ;\
99	> echo 'file svcsock.c line 1563 +p' > /proc/dprintk
100	
101	or even like:
102	
103	nullarbor:~ # (
104	> echo 'file svcsock.c line 1603 +p' ;\
105	> echo 'file svcsock.c line 1563 +p' ;\
106	> ) > /proc/dprintk
107	
108	At the syntactical level, a command comprises a sequence of match
109	specifications, followed by a flags change specification.
110	
111	command ::= match-spec* flags-spec
112	
113	The match-spec's are used to choose a subset of the known dprintk()
114	callsites to which to apply the flags-spec.  Think of them as a query
115	with implicit ANDs between each pair.  Note that an empty list of
116	match-specs is possible, but is not very useful because it will not
117	match any debug statement callsites.
118	
119	A match specification comprises a keyword, which controls the attribute
120	of the callsite to be compared, and a value to compare against.  Possible
121	keywords are:
122	
123	match-spec ::= 'func' string |
124		       'file' string |
125		       'module' string |
126		       'format' string |
127		       'line' line-range
128	
129	line-range ::= lineno |
130		       '-'lineno |
131		       lineno'-' |
132		       lineno'-'lineno
133	// Note: line-range cannot contain space, e.g.
134	// "1-30" is valid range but "1 - 30" is not.
135	
136	lineno ::= unsigned-int
137	
138	The meanings of each keyword are:
139	
140	func
141	    The given string is compared against the function name
142	    of each callsite.  Example:
143	
144	    func svc_tcp_accept
145	
146	file
147	    The given string is compared against either the full
148	    pathname or the basename of the source file of each
149	    callsite.  Examples:
150	
151	    file svcsock.c
152	    file /usr/src/packages/BUILD/sgi-enhancednfs-1.4/default/net/sunrpc/svcsock.c
153	
154	module
155	    The given string is compared against the module name
156	    of each callsite.  The module name is the string as
157	    seen in "lsmod", i.e. without the directory or the .ko
158	    suffix and with '-' changed to '_'.  Examples:
159	
160	    module sunrpc
161	    module nfsd
162	
163	format
164	    The given string is searched for in the dynamic debug format
165	    string.  Note that the string does not need to match the
166	    entire format, only some part.  Whitespace and other
167	    special characters can be escaped using C octal character
168	    escape \ooo notation, e.g. the space character is \040.
169	    Alternatively, the string can be enclosed in double quote
170	    characters (") or single quote characters (').
171	    Examples:
172	
173	    format svcrdma:	    // many of the NFS/RDMA server dprintks
174	    format readahead	    // some dprintks in the readahead cache
175	    format nfsd:\040SETATTR // one way to match a format with whitespace
176	    format "nfsd: SETATTR"  // a neater way to match a format with whitespace
177	    format 'nfsd: SETATTR'  // yet another way to match a format with whitespace
178	
179	line
180	    The given line number or range of line numbers is compared
181	    against the line number of each dprintk() callsite.  A single
182	    line number matches the callsite line number exactly.  A
183	    range of line numbers matches any callsite between the first
184	    and last line number inclusive.  An empty first number means
185	    the first line in the file, an empty line number means the
186	    last number in the file.  Examples:
187	
188	    line 1603	    // exactly line 1603
189	    line 1600-1605  // the six lines from line 1600 to line 1605
190	    line -1605	    // the 1605 lines from line 1 to line 1605
191	    line 1600-	    // all lines from line 1600 to the end of the file
192	
193	The flags specification comprises a change operation followed
194	by one or more flag characters.  The change operation is one
195	of the characters:
196	
197	-
198	    remove the given flags
199	
200	+
201	    add the given flags
202	
203	=
204	    set the flags to the given flags
205	
206	The flags are:
207	
208	f
209	    Include the function name in the printed message
210	l
211	    Include line number in the printed message
212	m
213	    Include module name in the printed message
214	p
215	    Causes a printk() message to be emitted to dmesg
216	t
217	    Include thread ID in messages not generated from interrupt context
218	
219	Note the regexp ^[-+=][flmpt]+$ matches a flags specification.
220	Note also that there is no convenient syntax to remove all
221	the flags at once, you need to use "-flmpt".
222	
223	
224	Debug messages during boot process
225	==================================
226	
227	To be able to activate debug messages during the boot process,
228	even before userspace and debugfs exists, use the boot parameter:
229	ddebug_query="QUERY"
230	
231	QUERY follows the syntax described above, but must not exceed 1023
232	characters. The enablement of debug messages is done as an arch_initcall.
233	Thus you can enable debug messages in all code processed after this
234	arch_initcall via this boot parameter.
235	On an x86 system for example ACPI enablement is a subsys_initcall and
236	ddebug_query="file ec.c +p"
237	will show early Embedded Controller transactions during ACPI setup if
238	your machine (typically a laptop) has an Embedded Controller.
239	PCI (or other devices) initialization also is a hot candidate for using
240	this boot parameter for debugging purposes.
241	
242	
243	Examples
244	========
245	
246	// enable the message at line 1603 of file svcsock.c
247	nullarbor:~ # echo -n 'file svcsock.c line 1603 +p' >
248					<debugfs>/dynamic_debug/control
249	
250	// enable all the messages in file svcsock.c
251	nullarbor:~ # echo -n 'file svcsock.c +p' >
252					<debugfs>/dynamic_debug/control
253	
254	// enable all the messages in the NFS server module
255	nullarbor:~ # echo -n 'module nfsd +p' >
256					<debugfs>/dynamic_debug/control
257	
258	// enable all 12 messages in the function svc_process()
259	nullarbor:~ # echo -n 'func svc_process +p' >
260					<debugfs>/dynamic_debug/control
261	
262	// disable all 12 messages in the function svc_process()
263	nullarbor:~ # echo -n 'func svc_process -p' >
264					<debugfs>/dynamic_debug/control
265	
266	// enable messages for NFS calls READ, READLINK, READDIR and READDIR+.
267	nullarbor:~ # echo -n 'format "nfsd: READ" +p' >
268					<debugfs>/dynamic_debug/control
Hide Line Numbers
About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog
Information is copyright its respective author. All material is available from the Linux Kernel Source distributed under a GPL License. This page is provided as a free service by mjmwired.net.