About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Documentation / filesystems / nfs / fault_injection.txt


Based on kernel version 4.16.1. Page generated on 2018-04-09 11:53 EST.

1	
2	Fault Injection
3	===============
4	Fault injection is a method for forcing errors that may not normally occur, or
5	may be difficult to reproduce.  Forcing these errors in a controlled environment
6	can help the developer find and fix bugs before their code is shipped in a
7	production system.  Injecting an error on the Linux NFS server will allow us to
8	observe how the client reacts and if it manages to recover its state correctly.
9	
10	NFSD_FAULT_INJECTION must be selected when configuring the kernel to use this
11	feature.
12	
13	
14	Using Fault Injection
15	=====================
16	On the client, mount the fault injection server through NFS v4.0+ and do some
17	work over NFS (open files, take locks, ...).
18	
19	On the server, mount the debugfs filesystem to <debug_dir> and ls
20	<debug_dir>/nfsd.  This will show a list of files that will be used for
21	injecting faults on the NFS server.  As root, write a number n to the file
22	corresponding to the action you want the server to take.  The server will then
23	process the first n items it finds.  So if you want to forget 5 locks, echo '5'
24	to <debug_dir>/nfsd/forget_locks.  A value of 0 will tell the server to forget
25	all corresponding items.  A log message will be created containing the number
26	of items forgotten (check dmesg).
27	
28	Go back to work on the client and check if the client recovered from the error
29	correctly.
30	
31	
32	Available Faults
33	================
34	forget_clients:
35	     The NFS server keeps a list of clients that have placed a mount call.  If
36	     this list is cleared, the server will have no knowledge of who the client
37	     is, forcing the client to reauthenticate with the server.
38	
39	forget_openowners:
40	     The NFS server keeps a list of what files are currently opened and who
41	     they were opened by.  Clearing this list will force the client to reopen
42	     its files.
43	
44	forget_locks:
45	     The NFS server keeps a list of what files are currently locked in the VFS.
46	     Clearing this list will force the client to reclaim its locks (files are
47	     unlocked through the VFS as they are cleared from this list).
48	
49	forget_delegations:
50	     A delegation is used to assure the client that a file, or part of a file,
51	     has not changed since the delegation was awarded.  Clearing this list will
52	     force the client to reacquire its delegation before accessing the file
53	     again.
54	
55	recall_delegations:
56	     Delegations can be recalled by the server when another client attempts to
57	     access a file.  This test will notify the client that its delegation has
58	     been revoked, forcing the client to reacquire the delegation before using
59	     the file again.
60	
61	
62	tools/nfs/inject_faults.sh script
63	=================================
64	This script has been created to ease the fault injection process.  This script
65	will detect the mounted debugfs directory and write to the files located there
66	based on the arguments passed by the user.  For example, running
67	`inject_faults.sh forget_locks 1` as root will instruct the server to forget
68	one lock.  Running `inject_faults forget_locks` will instruct the server to
69	forgetall locks.
Hide Line Numbers


About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog