About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Documentation / virtual / uml / UserModeLinux-HOWTO.txt


Based on kernel version 4.16.1. Page generated on 2018-04-09 11:53 EST.

1	  User Mode Linux HOWTO
2	  User Mode Linux Core Team
3	  Mon Nov 18 14:16:16 EST 2002
4	
5	  This document describes the use and abuse of Jeff Dike's User Mode
6	  Linux: a port of the Linux kernel as a normal Intel Linux process.
7	  ______________________________________________________________________
8	
9	  Table of Contents
10	
11	  1. Introduction
12	
13	     1.1 How is User Mode Linux Different?
14	     1.2 Why Would I Want User Mode Linux?
15	
16	  2. Compiling the kernel and modules
17	
18	     2.1 Compiling the kernel
19	     2.2 Compiling and installing kernel modules
20	     2.3 Compiling and installing uml_utilities
21	
22	  3. Running UML and logging in
23	
24	     3.1 Running UML
25	     3.2 Logging in
26	     3.3 Examples
27	
28	  4. UML on 2G/2G hosts
29	
30	     4.1 Introduction
31	     4.2 The problem
32	     4.3 The solution
33	
34	  5. Setting up serial lines and consoles
35	
36	     5.1 Specifying the device
37	     5.2 Specifying the channel
38	     5.3 Examples
39	
40	  6. Setting up the network
41	
42	     6.1 General setup
43	     6.2 Userspace daemons
44	     6.3 Specifying ethernet addresses
45	     6.4 UML interface setup
46	     6.5 Multicast
47	     6.6 TUN/TAP with the uml_net helper
48	     6.7 TUN/TAP with a preconfigured tap device
49	     6.8 Ethertap
50	     6.9 The switch daemon
51	     6.10 Slip
52	     6.11 Slirp
53	     6.12 pcap
54	     6.13 Setting up the host yourself
55	
56	  7. Sharing Filesystems between Virtual Machines
57	
58	     7.1 A warning
59	     7.2 Using layered block devices
60	     7.3 Note!
61	     7.4 Another warning
62	     7.5 uml_moo : Merging a COW file with its backing file
63	
64	  8. Creating filesystems
65	
66	     8.1 Create the filesystem file
67	     8.2 Assign the file to a UML device
68	     8.3 Creating and mounting the filesystem
69	
70	  9. Host file access
71	
72	     9.1 Using hostfs
73	     9.2 hostfs as the root filesystem
74	     9.3 Building hostfs
75	
76	  10. The Management Console
77	     10.1 version
78	     10.2 halt and reboot
79	     10.3 config
80	     10.4 remove
81	     10.5 sysrq
82	     10.6 help
83	     10.7 cad
84	     10.8 stop
85	     10.9 go
86	
87	  11. Kernel debugging
88	
89	     11.1 Starting the kernel under gdb
90	     11.2 Examining sleeping processes
91	     11.3 Running ddd on UML
92	     11.4 Debugging modules
93	     11.5 Attaching gdb to the kernel
94	     11.6 Using alternate debuggers
95	
96	  12. Kernel debugging examples
97	
98	     12.1 The case of the hung fsck
99	     12.2 Episode 2: The case of the hung fsck
100	
101	  13. What to do when UML doesn't work
102	
103	     13.1 Strange compilation errors when you build from source
104	     13.2 (obsolete)
105	     13.3 A variety of panics and hangs with /tmp on a reiserfs  filesystem
106	     13.4 The compile fails with errors about conflicting types for 'open', 'dup', and 'waitpid'
107	     13.5 UML doesn't work when /tmp is an NFS filesystem
108	     13.6 UML hangs on boot when compiled with gprof support
109	     13.7 syslogd dies with a SIGTERM on startup
110	     13.8 TUN/TAP networking doesn't work on a 2.4 host
111	     13.9 You can network to the host but not to other machines on the net
112	     13.10 I have no root and I want to scream
113	     13.11 UML build conflict between ptrace.h and ucontext.h
114	     13.12 The UML BogoMips is exactly half the host's BogoMips
115	     13.13 When you run UML, it immediately segfaults
116	     13.14 xterms appear, then immediately disappear
117	     13.15 Any other panic, hang, or strange behavior
118	
119	  14. Diagnosing Problems
120	
121	     14.1 Case 1 : Normal kernel panics
122	     14.2 Case 2 : Tracing thread panics
123	     14.3 Case 3 : Tracing thread panics caused by other threads
124	     14.4 Case 4 : Hangs
125	
126	  15. Thanks
127	
128	     15.1 Code and Documentation
129	     15.2 Flushing out bugs
130	     15.3 Buglets and clean-ups
131	     15.4 Case Studies
132	     15.5 Other contributions
133	
134	
135	  ______________________________________________________________________
136	
137	  1.  Introduction
138	
139	  Welcome to User Mode Linux.  It's going to be fun.
140	
141	
142	
143	  1.1.  How is User Mode Linux Different?
144	
145	  Normally, the Linux Kernel talks straight to your hardware (video
146	  card, keyboard, hard drives, etc), and any programs which run ask the
147	  kernel to operate the hardware, like so:
148	
149	
150	
151	         +-----------+-----------+----+
152	         | Process 1 | Process 2 | ...|
153	         +-----------+-----------+----+
154	         |       Linux Kernel         |
155	         +----------------------------+
156	         |         Hardware           |
157	         +----------------------------+
158	
159	
160	
161	
162	  The User Mode Linux Kernel is different; instead of talking to the
163	  hardware, it talks to a `real' Linux kernel (called the `host kernel'
164	  from now on), like any other program.  Programs can then run inside
165	  User-Mode Linux as if they were running under a normal kernel, like
166	  so:
167	
168	
169	
170	                     +----------------+
171	                     | Process 2 | ...|
172	         +-----------+----------------+
173	         | Process 1 | User-Mode Linux|
174	         +----------------------------+
175	         |       Linux Kernel         |
176	         +----------------------------+
177	         |         Hardware           |
178	         +----------------------------+
179	
180	
181	
182	
183	
184	  1.2.  Why Would I Want User Mode Linux?
185	
186	
187	  1. If User Mode Linux crashes, your host kernel is still fine.
188	
189	  2. You can run a usermode kernel as a non-root user.
190	
191	  3. You can debug the User Mode Linux like any normal process.
192	
193	  4. You can run gprof (profiling) and gcov (coverage testing).
194	
195	  5. You can play with your kernel without breaking things.
196	
197	  6. You can use it as a sandbox for testing new apps.
198	
199	  7. You can try new development kernels safely.
200	
201	  8. You can run different distributions simultaneously.
202	
203	  9. It's extremely fun.
204	
205	
206	
207	
208	
209	  2.  Compiling the kernel and modules
210	
211	
212	
213	
214	  2.1.  Compiling the kernel
215	
216	
217	  Compiling the user mode kernel is just like compiling any other
218	  kernel.  Let's go through the steps, using 2.4.0-prerelease (current
219	  as of this writing) as an example:
220	
221	
222	  1. Download the latest UML patch from
223	
224	     the download page <http://user-mode-linux.sourceforge.net/
225	
226	     In this example, the file is uml-patch-2.4.0-prerelease.bz2.
227	
228	
229	  2. Download the matching kernel from your favourite kernel mirror,
230	     such as:
231	
232	     ftp://ftp.ca.kernel.org/pub/kernel/v2.4/linux-2.4.0-prerelease.tar.bz2
233	     <ftp://ftp.ca.kernel.org/pub/kernel/v2.4/linux-2.4.0-prerelease.tar.bz2>
234	     .
235	
236	
237	  3. Make a directory and unpack the kernel into it.
238	
239	
240	
241	       host%
242	       mkdir ~/uml
243	
244	
245	
246	
247	
248	
249	       host%
250	       cd ~/uml
251	
252	
253	
254	
255	
256	
257	       host%
258	       tar -xzvf linux-2.4.0-prerelease.tar.bz2
259	
260	
261	
262	
263	
264	
265	  4. Apply the patch using
266	
267	
268	
269	       host%
270	       cd ~/uml/linux
271	
272	
273	
274	       host%
275	       bzcat uml-patch-2.4.0-prerelease.bz2 | patch -p1
276	
277	
278	
279	
280	
281	
282	  5. Run your favorite config; `make xconfig ARCH=um' is the most
283	     convenient.  `make config ARCH=um' and 'make menuconfig ARCH=um'
284	     will work as well.  The defaults will give you a useful kernel.  If
285	     you want to change something, go ahead, it probably won't hurt
286	     anything.
287	
288	
289	     Note:  If the host is configured with a 2G/2G address space split
290	     rather than the usual 3G/1G split, then the packaged UML binaries
291	     will not run.  They will immediately segfault.  See ``UML on 2G/2G
292	     hosts''  for the scoop on running UML on your system.
293	
294	
295	
296	  6. Finish with `make linux ARCH=um': the result is a file called
297	     `linux' in the top directory of your source tree.
298	
299	  Make sure that you don't build this kernel in /usr/src/linux.  On some
300	  distributions, /usr/include/asm is a link into this pool.  The user-
301	  mode build changes the other end of that link, and things that include
302	  <asm/anything.h> stop compiling.
303	
304	  The sources are also available from cvs at the project's cvs page,
305	  which has directions on getting the sources. You can also browse the
306	  CVS pool from there.
307	
308	  If you get the CVS sources, you will have to check them out into an
309	  empty directory. You will then have to copy each file into the
310	  corresponding directory in the appropriate kernel pool.
311	
312	  If you don't have the latest kernel pool, you can get the
313	  corresponding user-mode sources with
314	
315	
316	       host% cvs co -r v_2_3_x linux
317	
318	
319	
320	
321	  where 'x' is the version in your pool. Note that you will not get the
322	  bug fixes and enhancements that have gone into subsequent releases.
323	
324	
325	  2.2.  Compiling and installing kernel modules
326	
327	  UML modules are built in the same way as the native kernel (with the
328	  exception of the 'ARCH=um' that you always need for UML):
329	
330	
331	       host% make modules ARCH=um
332	
333	
334	
335	
336	  Any modules that you want to load into this kernel need to be built in
337	  the user-mode pool.  Modules from the native kernel won't work.
338	
339	  You can install them by using ftp or something to copy them into the
340	  virtual machine and dropping them into /lib/modules/`uname -r`.
341	
342	  You can also get the kernel build process to install them as follows:
343	
344	  1. with the kernel not booted, mount the root filesystem in the top
345	     level of the kernel pool:
346	
347	
348	       host% mount root_fs mnt -o loop
349	
350	
351	
352	
353	
354	
355	  2. run
356	
357	
358	       host%
359	       make modules_install INSTALL_MOD_PATH=`pwd`/mnt ARCH=um
360	
361	
362	
363	
364	
365	
366	  3. unmount the filesystem
367	
368	
369	       host% umount mnt
370	
371	
372	
373	
374	
375	
376	  4. boot the kernel on it
377	
378	
379	  When the system is booted, you can use insmod as usual to get the
380	  modules into the kernel.  A number of things have been loaded into UML
381	  as modules, especially filesystems and network protocols and filters,
382	  so most symbols which need to be exported probably already are.
383	  However, if you do find symbols that need exporting, let  us
384	  <http://user-mode-linux.sourceforge.net/>  know, and
385	  they'll be "taken care of".
386	
387	
388	
389	  2.3.  Compiling and installing uml_utilities
390	
391	  Many features of the UML kernel require a user-space helper program,
392	  so a uml_utilities package is distributed separately from the kernel
393	  patch which provides these helpers. Included within this is:
394	
395	  o  port-helper - Used by consoles which connect to xterms or ports
396	
397	  o  tunctl - Configuration tool to create and delete tap devices
398	
399	  o  uml_net - Setuid binary for automatic tap device configuration
400	
401	  o  uml_switch - User-space virtual switch required for daemon
402	     transport
403	
404	     The uml_utilities tree is compiled with:
405	
406	
407	       host#
408	       make && make install
409	
410	
411	
412	
413	  Note that UML kernel patches may require a specific version of the
414	  uml_utilities distribution. If you don't keep up with the mailing
415	  lists, ensure that you have the latest release of uml_utilities if you
416	  are experiencing problems with your UML kernel, particularly when
417	  dealing with consoles or command-line switches to the helper programs
418	
419	
420	
421	
422	
423	
424	
425	
426	  3.  Running UML and logging in
427	
428	
429	
430	  3.1.  Running UML
431	
432	  It runs on 2.2.15 or later, and all 2.4 kernels.
433	
434	
435	  Booting UML is straightforward.  Simply run 'linux': it will try to
436	  mount the file `root_fs' in the current directory.  You do not need to
437	  run it as root.  If your root filesystem is not named `root_fs', then
438	  you need to put a `ubd0=root_fs_whatever' switch on the linux command
439	  line.
440	
441	
442	  You will need a filesystem to boot UML from.  There are a number
443	  available for download from  here  <http://user-mode-
444	  linux.sourceforge.net/> .  There are also  several tools
445	  <http://user-mode-linux.sourceforge.net/>  which can be
446	  used to generate UML-compatible filesystem images from media.
447	  The kernel will boot up and present you with a login prompt.
448	
449	
450	  Note:  If the host is configured with a 2G/2G address space split
451	  rather than the usual 3G/1G split, then the packaged UML binaries will
452	  not run.  They will immediately segfault.  See ``UML on 2G/2G hosts''
453	  for the scoop on running UML on your system.
454	
455	
456	
457	  3.2.  Logging in
458	
459	
460	
461	  The prepackaged filesystems have a root account with password 'root'
462	  and a user account with password 'user'.  The login banner will
463	  generally tell you how to log in.  So, you log in and you will find
464	  yourself inside a little virtual machine. Our filesystems have a
465	  variety of commands and utilities installed (and it is fairly easy to
466	  add more), so you will have a lot of tools with which to poke around
467	  the system.
468	
469	  There are a couple of other ways to log in:
470	
471	  o  On a virtual console
472	
473	
474	
475	     Each virtual console that is configured (i.e. the device exists in
476	     /dev and /etc/inittab runs a getty on it) will come up in its own
477	     xterm.  If you get tired of the xterms, read ``Setting up serial
478	     lines and consoles''  to see how to attach the consoles to
479	     something else, like host ptys.
480	
481	
482	
483	  o  Over the serial line
484	
485	
486	     In the boot output, find a line that looks like:
487	
488	
489	
490	       serial line 0 assigned pty /dev/ptyp1
491	
492	
493	
494	
495	  Attach your favorite terminal program to the corresponding tty.  I.e.
496	  for minicom, the command would be
497	
498	
499	       host% minicom -o -p /dev/ttyp1
500	
501	
502	
503	
504	
505	
506	  o  Over the net
507	
508	
509	     If the network is running, then you can telnet to the virtual
510	     machine and log in to it.  See ``Setting up the network''  to learn
511	     about setting up a virtual network.
512	
513	  When you're done using it, run halt, and the kernel will bring itself
514	  down and the process will exit.
515	
516	
517	  3.3.  Examples
518	
519	  Here are some examples of UML in action:
520	
521	  o  A login session <http://user-mode-linux.sourceforge.net/login.html>
522	
523	  o  A virtual network <http://user-mode-linux.sourceforge.net/net.html>
524	
525	
526	
527	
528	
529	
530	
531	  4.  UML on 2G/2G hosts
532	
533	
534	
535	
536	  4.1.  Introduction
537	
538	
539	  Most Linux machines are configured so that the kernel occupies the
540	  upper 1G (0xc0000000 - 0xffffffff) of the 4G address space and
541	  processes use the lower 3G (0x00000000 - 0xbfffffff).  However, some
542	  machine are configured with a 2G/2G split, with the kernel occupying
543	  the upper 2G (0x80000000 - 0xffffffff) and processes using the lower
544	  2G (0x00000000 - 0x7fffffff).
545	
546	
547	
548	
549	  4.2.  The problem
550	
551	
552	  The prebuilt UML binaries on this site will not run on 2G/2G hosts
553	  because UML occupies the upper .5G of the 3G process address space
554	  (0xa0000000 - 0xbfffffff).  Obviously, on 2G/2G hosts, this is right
555	  in the middle of the kernel address space, so UML won't even load - it
556	  will immediately segfault.
557	
558	
559	
560	
561	  4.3.  The solution
562	
563	
564	  The fix for this is to rebuild UML from source after enabling
565	  CONFIG_HOST_2G_2G (under 'General Setup').  This will cause UML to
566	  load itself in the top .5G of that smaller process address space,
567	  where it will run fine.  See ``Compiling the kernel and modules''  if
568	  you need help building UML from source.
569	
570	
571	
572	
573	
574	
575	
576	
577	
578	
579	  5.  Setting up serial lines and consoles
580	
581	
582	  It is possible to attach UML serial lines and consoles to many types
583	  of host I/O channels by specifying them on the command line.
584	
585	
586	  You can attach them to host ptys, ttys, file descriptors, and ports.
587	  This allows you to do things like
588	
589	  o  have a UML console appear on an unused host console,
590	
591	  o  hook two virtual machines together by having one attach to a pty
592	     and having the other attach to the corresponding tty
593	
594	  o  make a virtual machine accessible from the net by attaching a
595	     console to a port on the host.
596	
597	
598	  The general format of the command line option is device=channel.
599	
600	
601	
602	  5.1.  Specifying the device
603	
604	  Devices are specified with "con" or "ssl" (console or serial line,
605	  respectively), optionally with a device number if you are talking
606	  about a specific device.
607	
608	
609	  Using just "con" or "ssl" describes all of the consoles or serial
610	  lines.  If you want to talk about console #3 or serial line #10, they
611	  would be "con3" and "ssl10", respectively.
612	
613	
614	  A specific device name will override a less general "con=" or "ssl=".
615	  So, for example, you can assign a pty to each of the serial lines
616	  except for the first two like this:
617	
618	
619	        ssl=pty ssl0=tty:/dev/tty0 ssl1=tty:/dev/tty1
620	
621	
622	
623	
624	  The specificity of the device name is all that matters; order on the
625	  command line is irrelevant.
626	
627	
628	
629	  5.2.  Specifying the channel
630	
631	  There are a number of different types of channels to attach a UML
632	  device to, each with a different way of specifying exactly what to
633	  attach to.
634	
635	  o  pseudo-terminals - device=pty pts terminals - device=pts
636	
637	
638	     This will cause UML to allocate a free host pseudo-terminal for the
639	     device.  The terminal that it got will be announced in the boot
640	     log.  You access it by attaching a terminal program to the
641	     corresponding tty:
642	
643	  o  screen /dev/pts/n
644	
645	  o  screen /dev/ttyxx
646	
647	  o  minicom -o -p /dev/ttyxx - minicom seems not able to handle pts
648	     devices
649	
650	  o  kermit - start it up, 'open' the device, then 'connect'
651	
652	
653	
654	
655	
656	  o  terminals - device=tty:tty device file
657	
658	
659	     This will make UML attach the device to the specified tty (i.e
660	
661	
662	        con1=tty:/dev/tty3
663	
664	
665	
666	
667	  will attach UML's console 1 to the host's /dev/tty3).  If the tty that
668	  you specify is the slave end of a tty/pty pair, something else must
669	  have already opened the corresponding pty in order for this to work.
670	
671	
672	
673	
674	
675	  o  xterms - device=xterm
676	
677	
678	     UML will run an xterm and the device will be attached to it.
679	
680	
681	
682	
683	
684	  o  Port - device=port:port number
685	
686	
687	     This will attach the UML devices to the specified host port.
688	     Attaching console 1 to the host's port 9000 would be done like
689	     this:
690	
691	
692	        con1=port:9000
693	
694	
695	
696	
697	  Attaching all the serial lines to that port would be done similarly:
698	
699	
700	        ssl=port:9000
701	
702	
703	
704	
705	  You access these devices by telnetting to that port.  Each active tel-
706	  net session gets a different device.  If there are more telnets to a
707	  port than UML devices attached to it, then the extra telnet sessions
708	  will block until an existing telnet detaches, or until another device
709	  becomes active (i.e. by being activated in /etc/inittab).
710	
711	  This channel has the advantage that you can both attach multiple UML
712	  devices to it and know how to access them without reading the UML boot
713	  log.  It is also unique in allowing access to a UML from remote
714	  machines without requiring that the UML be networked.  This could be
715	  useful in allowing public access to UMLs because they would be
716	  accessible from the net, but wouldn't need any kind of network
717	  filtering or access control because they would have no network access.
718	
719	
720	  If you attach the main console to a portal, then the UML boot will
721	  appear to hang.  In reality, it's waiting for a telnet to connect, at
722	  which point the boot will proceed.
723	
724	
725	
726	
727	
728	  o  already-existing file descriptors - device=file descriptor
729	
730	
731	     If you set up a file descriptor on the UML command line, you can
732	     attach a UML device to it.  This is most commonly used to put the
733	     main console back on stdin and stdout after assigning all the other
734	     consoles to something else:
735	
736	
737	        con0=fd:0,fd:1 con=pts
738	
739	
740	
741	
742	
743	
744	
745	
746	  o  Nothing - device=null
747	
748	
749	     This allows the device to be opened, in contrast to 'none', but
750	     reads will block, and writes will succeed and the data will be
751	     thrown out.
752	
753	
754	
755	
756	
757	  o  None - device=none
758	
759	
760	     This causes the device to disappear.
761	
762	
763	
764	  You can also specify different input and output channels for a device
765	  by putting a comma between them:
766	
767	
768	        ssl3=tty:/dev/tty2,xterm
769	
770	
771	
772	
773	  will cause serial line 3 to accept input on the host's /dev/tty2 and
774	  display output on an xterm.  That's a silly example - the most common
775	  use of this syntax is to reattach the main console to stdin and stdout
776	  as shown above.
777	
778	
779	  If you decide to move the main console away from stdin/stdout, the
780	  initial boot output will appear in the terminal that you're running
781	  UML in.  However, once the console driver has been officially
782	  initialized, then the boot output will start appearing wherever you
783	  specified that console 0 should be.  That device will receive all
784	  subsequent output.
785	
786	
787	
788	  5.3.  Examples
789	
790	  There are a number of interesting things you can do with this
791	  capability.
792	
793	
794	  First, this is how you get rid of those bleeding console xterms by
795	  attaching them to host ptys:
796	
797	
798	        con=pty con0=fd:0,fd:1
799	
800	
801	
802	
803	  This will make a UML console take over an unused host virtual console,
804	  so that when you switch to it, you will see the UML login prompt
805	  rather than the host login prompt:
806	
807	
808	        con1=tty:/dev/tty6
809	
810	
811	
812	
813	  You can attach two virtual machines together with what amounts to a
814	  serial line as follows:
815	
816	  Run one UML with a serial line attached to a pty -
817	
818	
819	        ssl1=pty
820	
821	
822	
823	
824	  Look at the boot log to see what pty it got (this example will assume
825	  that it got /dev/ptyp1).
826	
827	  Boot the other UML with a serial line attached to the corresponding
828	  tty -
829	
830	
831	        ssl1=tty:/dev/ttyp1
832	
833	
834	
835	
836	  Log in, make sure that it has no getty on that serial line, attach a
837	  terminal program like minicom to it, and you should see the login
838	  prompt of the other virtual machine.
839	
840	
841	  6.  Setting up the network
842	
843	
844	
845	  This page describes how to set up the various transports and to
846	  provide a UML instance with network access to the host, other machines
847	  on the local net, and the rest of the net.
848	
849	
850	  As of 2.4.5, UML networking has been completely redone to make it much
851	  easier to set up, fix bugs, and add new features.
852	
853	
854	  There is a new helper, uml_net, which does the host setup that
855	  requires root privileges.
856	
857	
858	  There are currently five transport types available for a UML virtual
859	  machine to exchange packets with other hosts:
860	
861	  o  ethertap
862	
863	  o  TUN/TAP
864	
865	  o  Multicast
866	
867	  o  a switch daemon
868	
869	  o  slip
870	
871	  o  slirp
872	
873	  o  pcap
874	
875	     The TUN/TAP, ethertap, slip, and slirp transports allow a UML
876	     instance to exchange packets with the host.  They may be directed
877	     to the host or the host may just act as a router to provide access
878	     to other physical or virtual machines.
879	
880	
881	  The pcap transport is a synthetic read-only interface, using the
882	  libpcap binary to collect packets from interfaces on the host and
883	  filter them.  This is useful for building preconfigured traffic
884	  monitors or sniffers.
885	
886	
887	  The daemon and multicast transports provide a completely virtual
888	  network to other virtual machines.  This network is completely
889	  disconnected from the physical network unless one of the virtual
890	  machines on it is acting as a gateway.
891	
892	
893	  With so many host transports, which one should you use?  Here's when
894	  you should use each one:
895	
896	  o  ethertap - if you want access to the host networking and it is
897	     running 2.2
898	
899	  o  TUN/TAP - if you want access to the host networking and it is
900	     running 2.4.  Also, the TUN/TAP transport is able to use a
901	     preconfigured device, allowing it to avoid using the setuid uml_net
902	     helper, which is a security advantage.
903	
904	  o  Multicast - if you want a purely virtual network and you don't want
905	     to set up anything but the UML
906	
907	  o  a switch daemon - if you want a purely virtual network and you
908	     don't mind running the daemon in order to get somewhat better
909	     performance
910	
911	  o  slip - there is no particular reason to run the slip backend unless
912	     ethertap and TUN/TAP are just not available for some reason
913	
914	  o  slirp - if you don't have root access on the host to setup
915	     networking, or if you don't want to allocate an IP to your UML
916	
917	  o  pcap - not much use for actual network connectivity, but great for
918	     monitoring traffic on the host
919	
920	     Ethertap is available on 2.4 and works fine.  TUN/TAP is preferred
921	     to it because it has better performance and ethertap is officially
922	     considered obsolete in 2.4.  Also, the root helper only needs to
923	     run occasionally for TUN/TAP, rather than handling every packet, as
924	     it does with ethertap.  This is a slight security advantage since
925	     it provides fewer opportunities for a nasty UML user to somehow
926	     exploit the helper's root privileges.
927	
928	
929	  6.1.  General setup
930	
931	  First, you must have the virtual network enabled in your UML.  If are
932	  running a prebuilt kernel from this site, everything is already
933	  enabled.  If you build the kernel yourself, under the "Network device
934	  support" menu, enable "Network device support", and then the three
935	  transports.
936	
937	
938	  The next step is to provide a network device to the virtual machine.
939	  This is done by describing it on the kernel command line.
940	
941	  The general format is
942	
943	
944	       eth <n> = <transport> , <transport args>
945	
946	
947	
948	
949	  For example, a virtual ethernet device may be attached to a host
950	  ethertap device as follows:
951	
952	
953	       eth0=ethertap,tap0,fe:fd:0:0:0:1,192.168.0.254
954	
955	
956	
957	
958	  This sets up eth0 inside the virtual machine to attach itself to the
959	  host /dev/tap0, assigns it an ethernet address, and assigns the host
960	  tap0 interface an IP address.
961	
962	
963	
964	  Note that the IP address you assign to the host end of the tap device
965	  must be different than the IP you assign to the eth device inside UML.
966	  If you are short on IPs and don't want to consume two per UML, then
967	  you can reuse the host's eth IP address for the host ends of the tap
968	  devices.  Internally, the UMLs must still get unique IPs for their eth
969	  devices.  You can also give the UMLs non-routable IPs (192.168.x.x or
970	  10.x.x.x) and have the host masquerade them.  This will let outgoing
971	  connections work, but incoming connections won't without more work,
972	  such as port forwarding from the host.
973	  Also note that when you configure the host side of an interface, it is
974	  only acting as a gateway.  It will respond to pings sent to it
975	  locally, but is not useful to do that since it's a host interface.
976	  You are not talking to the UML when you ping that interface and get a
977	  response.
978	
979	
980	  You can also add devices to a UML and remove them at runtime.  See the
981	  ``The Management Console''  page for details.
982	
983	
984	  The sections below describe this in more detail.
985	
986	
987	  Once you've decided how you're going to set up the devices, you boot
988	  UML, log in, configure the UML side of the devices, and set up routes
989	  to the outside world.  At that point, you will be able to talk to any
990	  other machines, physical or virtual, on the net.
991	
992	
993	  If ifconfig inside UML fails and the network refuses to come up, run
994	  tell you what went wrong.
995	
996	
997	
998	  6.2.  Userspace daemons
999	
1000	  You will likely need the setuid helper, or the switch daemon, or both.
1001	  They are both installed with the RPM and deb, so if you've installed
1002	  either, you can skip the rest of this section.
1003	
1004	
1005	  If not, then you need to check them out of CVS, build them, and
1006	  install them.  The helper is uml_net, in CVS /tools/uml_net, and the
1007	  daemon is uml_switch, in CVS /tools/uml_router.  They are both built
1008	  with a plain 'make'.  Both need to be installed in a directory that's
1009	  in your path - /usr/bin is recommend.  On top of that, uml_net needs
1010	  to be setuid root.
1011	
1012	
1013	
1014	  6.3.  Specifying ethernet addresses
1015	
1016	  Below, you will see that the TUN/TAP, ethertap, and daemon interfaces
1017	  allow you to specify hardware addresses for the virtual ethernet
1018	  devices.  This is generally not necessary.  If you don't have a
1019	  specific reason to do it, you probably shouldn't.  If one is not
1020	  specified on the command line, the driver will assign one based on the
1021	  device IP address.  It will provide the address fe:fd:nn:nn:nn:nn
1022	  where nn.nn.nn.nn is the device IP address.  This is nearly always
1023	  sufficient to guarantee a unique hardware address for the device.  A
1024	  couple of exceptions are:
1025	
1026	  o  Another set of virtual ethernet devices are on the same network and
1027	     they are assigned hardware addresses using a different scheme which
1028	     may conflict with the UML IP address-based scheme
1029	
1030	  o  You aren't going to use the device for IP networking, so you don't
1031	     assign the device an IP address
1032	
1033	     If you let the driver provide the hardware address, you should make
1034	     sure that the device IP address is known before the interface is
1035	     brought up.  So, inside UML, this will guarantee that:
1036	
1037	
1038	
1039	  UML#
1040	  ifconfig eth0 192.168.0.250 up
1041	
1042	
1043	
1044	
1045	  If you decide to assign the hardware address yourself, make sure that
1046	  the first byte of the address is even.  Addresses with an odd first
1047	  byte are broadcast addresses, which you don't want assigned to a
1048	  device.
1049	
1050	
1051	
1052	  6.4.  UML interface setup
1053	
1054	  Once the network devices have been described on the command line, you
1055	  should boot UML and log in.
1056	
1057	
1058	  The first thing to do is bring the interface up:
1059	
1060	
1061	       UML# ifconfig ethn ip-address up
1062	
1063	
1064	
1065	
1066	  You should be able to ping the host at this point.
1067	
1068	
1069	  To reach the rest of the world, you should set a default route to the
1070	  host:
1071	
1072	
1073	       UML# route add default gw host ip
1074	
1075	
1076	
1077	
1078	  Again, with host ip of 192.168.0.4:
1079	
1080	
1081	       UML# route add default gw 192.168.0.4
1082	
1083	
1084	
1085	
1086	  This page used to recommend setting a network route to your local net.
1087	  This is wrong, because it will cause UML to try to figure out hardware
1088	  addresses of the local machines by arping on the interface to the
1089	  host.  Since that interface is basically a single strand of ethernet
1090	  with two nodes on it (UML and the host) and arp requests don't cross
1091	  networks, they will fail to elicit any responses.  So, what you want
1092	  is for UML to just blindly throw all packets at the host and let it
1093	  figure out what to do with them, which is what leaving out the network
1094	  route and adding the default route does.
1095	
1096	
1097	  Note: If you can't communicate with other hosts on your physical
1098	  ethernet, it's probably because of a network route that's
1099	  automatically set up.  If you run 'route -n' and see a route that
1100	  looks like this:
1101	
1102	
1103	
1104	
1105	  Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
1106	  192.168.0.0     0.0.0.0         255.255.255.0   U     0      0      0   eth0
1107	
1108	
1109	
1110	
1111	  with a mask that's not 255.255.255.255, then replace it with a route
1112	  to your host:
1113	
1114	
1115	       UML#
1116	       route del -net 192.168.0.0 dev eth0 netmask 255.255.255.0
1117	
1118	
1119	
1120	
1121	
1122	
1123	       UML#
1124	       route add -host 192.168.0.4 dev eth0
1125	
1126	
1127	
1128	
1129	  This, plus the default route to the host, will allow UML to exchange
1130	  packets with any machine on your ethernet.
1131	
1132	
1133	
1134	  6.5.  Multicast
1135	
1136	  The simplest way to set up a virtual network between multiple UMLs is
1137	  to use the mcast transport.  This was written by Harald Welte and is
1138	  present in UML version 2.4.5-5um and later.  Your system must have
1139	  multicast enabled in the kernel and there must be a multicast-capable
1140	  network device on the host.  Normally, this is eth0, but if there is
1141	  no ethernet card on the host, then you will likely get strange error
1142	  messages when you bring the device up inside UML.
1143	
1144	
1145	  To use it, run two UMLs with
1146	
1147	
1148	        eth0=mcast
1149	
1150	
1151	
1152	
1153	  on their command lines.  Log in, configure the ethernet device in each
1154	  machine with different IP addresses:
1155	
1156	
1157	       UML1# ifconfig eth0 192.168.0.254
1158	
1159	
1160	
1161	
1162	
1163	
1164	       UML2# ifconfig eth0 192.168.0.253
1165	
1166	
1167	
1168	
1169	  and they should be able to talk to each other.
1170	
1171	  The full set of command line options for this transport are
1172	
1173	
1174	
1175	       ethn=mcast,ethernet address,multicast
1176	       address,multicast port,ttl
1177	
1178	
1179	
1180	
1181	  Harald's original README is here <http://user-mode-linux.source-
1182	  forge.net/>  and explains these in detail, as well as
1183	  some other issues.
1184	
1185	  There is also a related point-to-point only "ucast" transport.
1186	  This is useful when your network does not support multicast, and
1187	  all network connections are simple point to point links.
1188	
1189	  The full set of command line options for this transport are
1190	
1191	
1192	       ethn=ucast,ethernet address,remote address,listen port,remote port
1193	
1194	
1195	
1196	
1197	  6.6.  TUN/TAP with the uml_net helper
1198	
1199	  TUN/TAP is the preferred mechanism on 2.4 to exchange packets with the
1200	  host.  The TUN/TAP backend has been in UML since 2.4.9-3um.
1201	
1202	
1203	  The easiest way to get up and running is to let the setuid uml_net
1204	  helper do the host setup for you.  This involves insmod-ing the tun.o
1205	  module if necessary, configuring the device, and setting up IP
1206	  forwarding, routing, and proxy arp.  If you are new to UML networking,
1207	  do this first.  If you're concerned about the security implications of
1208	  the setuid helper, use it to get up and running, then read the next
1209	  section to see how to have UML use a preconfigured tap device, which
1210	  avoids the use of uml_net.
1211	
1212	
1213	  If you specify an IP address for the host side of the device, the
1214	  uml_net helper will do all necessary setup on the host - the only
1215	  requirement is that TUN/TAP be available, either built in to the host
1216	  kernel or as the tun.o module.
1217	
1218	  The format of the command line switch to attach a device to a TUN/TAP
1219	  device is
1220	
1221	
1222	       eth <n> =tuntap,,, <IP address>
1223	
1224	
1225	
1226	
1227	  For example, this argument will attach the UML's eth0 to the next
1228	  available tap device and assign an ethernet address to it based on its
1229	  IP address
1230	
1231	
1232	       eth0=tuntap,,,192.168.0.254
1233	
1234	
1235	
1236	
1237	
1238	
1239	  Note that the IP address that must be used for the eth device inside
1240	  UML is fixed by the routing and proxy arp that is set up on the
1241	  TUN/TAP device on the host.  You can use a different one, but it won't
1242	  work because reply packets won't reach the UML.  This is a feature.
1243	  It prevents a nasty UML user from doing things like setting the UML IP
1244	  to the same as the network's nameserver or mail server.
1245	
1246	
1247	  There are a couple potential problems with running the TUN/TAP
1248	  transport on a 2.4 host kernel
1249	
1250	  o  TUN/TAP seems not to work on 2.4.3 and earlier.  Upgrade the host
1251	     kernel or use the ethertap transport.
1252	
1253	  o  With an upgraded kernel, TUN/TAP may fail with
1254	
1255	
1256	       File descriptor in bad state
1257	
1258	
1259	
1260	
1261	  This is due to a header mismatch between the upgraded kernel and the
1262	  kernel that was originally installed on the machine.  The fix is to
1263	  make sure that /usr/src/linux points to the headers for the running
1264	  kernel.
1265	
1266	  These were pointed out by Tim Robinson <timro at trkr dot net> in
1267	  <http://www.geocrawler.com/> name="this uml-
1268	  user post"> .
1269	
1270	
1271	
1272	  6.7.  TUN/TAP with a preconfigured tap device
1273	
1274	  If you prefer not to have UML use uml_net (which is somewhat
1275	  insecure), with UML 2.4.17-11, you can set up a TUN/TAP device
1276	  beforehand.  The setup needs to be done as root, but once that's done,
1277	  there is no need for root assistance.  Setting up the device is done
1278	  as follows:
1279	
1280	  o  Create the device with tunctl (available from the UML utilities
1281	     tarball)
1282	
1283	
1284	
1285	
1286	       host#  tunctl -u uid
1287	
1288	
1289	
1290	
1291	  where uid is the user id or username that UML will be run as.  This
1292	  will tell you what device was created.
1293	
1294	  o  Configure the device IP (change IP addresses and device name to
1295	     suit)
1296	
1297	
1298	
1299	
1300	       host#  ifconfig tap0 192.168.0.254 up
1301	
1302	
1303	
1304	
1305	
1306	  o  Set up routing and arping if desired - this is my recipe, there are
1307	     other ways of doing the same thing
1308	
1309	
1310	       host#
1311	       bash -c 'echo 1 > /proc/sys/net/ipv4/ip_forward'
1312	
1313	       host#
1314	       route add -host 192.168.0.253 dev tap0
1315	
1316	
1317	
1318	
1319	
1320	
1321	       host#
1322	       bash -c 'echo 1 > /proc/sys/net/ipv4/conf/tap0/proxy_arp'
1323	
1324	
1325	
1326	
1327	
1328	
1329	       host#
1330	       arp -Ds 192.168.0.253 eth0 pub
1331	
1332	
1333	
1334	
1335	  Note that this must be done every time the host boots - this configu-
1336	  ration is not stored across host reboots.  So, it's probably a good
1337	  idea to stick it in an rc file.  An even better idea would be a little
1338	  utility which reads the information from a config file and sets up
1339	  devices at boot time.
1340	
1341	  o  Rather than using up two IPs and ARPing for one of them, you can
1342	     also provide direct access to your LAN by the UML by using a
1343	     bridge.
1344	
1345	
1346	       host#
1347	       brctl addbr br0
1348	
1349	
1350	
1351	
1352	
1353	
1354	       host#
1355	       ifconfig eth0 0.0.0.0 promisc up
1356	
1357	
1358	
1359	
1360	
1361	
1362	       host#
1363	       ifconfig tap0 0.0.0.0 promisc up
1364	
1365	
1366	
1367	
1368	
1369	
1370	       host#
1371	       ifconfig br0 192.168.0.1 netmask 255.255.255.0 up
1372	
1373	
1374	
1375	
1376	
1377	
1378	
1379	  host#
1380	  brctl stp br0 off
1381	
1382	
1383	
1384	
1385	
1386	
1387	       host#
1388	       brctl setfd br0 1
1389	
1390	
1391	
1392	
1393	
1394	
1395	       host#
1396	       brctl sethello br0 1
1397	
1398	
1399	
1400	
1401	
1402	
1403	       host#
1404	       brctl addif br0 eth0
1405	
1406	
1407	
1408	
1409	
1410	
1411	       host#
1412	       brctl addif br0 tap0
1413	
1414	
1415	
1416	
1417	  Note that 'br0' should be setup using ifconfig with the existing IP
1418	  address of eth0, as eth0 no longer has its own IP.
1419	
1420	  o
1421	
1422	
1423	     Also, the /dev/net/tun device must be writable by the user running
1424	     UML in order for the UML to use the device that's been configured
1425	     for it.  The simplest thing to do is
1426	
1427	
1428	       host#  chmod 666 /dev/net/tun
1429	
1430	
1431	
1432	
1433	  Making it world-writable looks bad, but it seems not to be
1434	  exploitable as a security hole.  However, it does allow anyone to cre-
1435	  ate useless tap devices (useless because they can't configure them),
1436	  which is a DOS attack.  A somewhat more secure alternative would to be
1437	  to create a group containing all the users who have preconfigured tap
1438	  devices and chgrp /dev/net/tun to that group with mode 664 or 660.
1439	
1440	
1441	  o  Once the device is set up, run UML with 'eth0=tuntap,device name'
1442	     (i.e. 'eth0=tuntap,tap0') on the command line (or do it with the
1443	     mconsole config command).
1444	
1445	  o  Bring the eth device up in UML and you're in business.
1446	
1447	     If you don't want that tap device any more, you can make it non-
1448	     persistent with
1449	
1450	
1451	       host#  tunctl -d tap device
1452	
1453	
1454	
1455	
1456	  Finally, tunctl has a -b (for brief mode) switch which causes it to
1457	  output only the name of the tap device it created.  This makes it
1458	  suitable for capture by a script:
1459	
1460	
1461	       host#  TAP=`tunctl -u 1000 -b`
1462	
1463	
1464	
1465	
1466	
1467	
1468	  6.8.  Ethertap
1469	
1470	  Ethertap is the general mechanism on 2.2 for userspace processes to
1471	  exchange packets with the kernel.
1472	
1473	
1474	
1475	  To use this transport, you need to describe the virtual network device
1476	  on the UML command line.  The general format for this is
1477	
1478	
1479	       eth <n> =ethertap, <device> , <ethernet address> , <tap IP address>
1480	
1481	
1482	
1483	
1484	  So, the previous example
1485	
1486	
1487	       eth0=ethertap,tap0,fe:fd:0:0:0:1,192.168.0.254
1488	
1489	
1490	
1491	
1492	  attaches the UML eth0 device to the host /dev/tap0, assigns it the
1493	  ethernet address fe:fd:0:0:0:1, and assigns the IP address
1494	  192.168.0.254 to the tap device.
1495	
1496	
1497	
1498	  The tap device is mandatory, but the others are optional.  If the
1499	  ethernet address is omitted, one will be assigned to it.
1500	
1501	
1502	  The presence of the tap IP address will cause the helper to run and do
1503	  whatever host setup is needed to allow the virtual machine to
1504	  communicate with the outside world.  If you're not sure you know what
1505	  you're doing, this is the way to go.
1506	
1507	
1508	  If it is absent, then you must configure the tap device and whatever
1509	  arping and routing you will need on the host.  However, even in this
1510	  case, the uml_net helper still needs to be in your path and it must be
1511	  setuid root if you're not running UML as root.  This is because the
1512	  tap device doesn't support SIGIO, which UML needs in order to use
1513	  something as a source of input.  So, the helper is used as a
1514	  convenient asynchronous IO thread.
1515	
1516	  If you're using the uml_net helper, you can ignore the following host
1517	  setup - uml_net will do it for you.  You just need to make sure you
1518	  have ethertap available, either built in to the host kernel or
1519	  available as a module.
1520	
1521	
1522	  If you want to set things up yourself, you need to make sure that the
1523	  appropriate /dev entry exists.  If it doesn't, become root and create
1524	  it as follows:
1525	
1526	
1527	       mknod /dev/tap <minor>  c 36  <minor>  + 16
1528	
1529	
1530	
1531	
1532	  For example, this is how to create /dev/tap0:
1533	
1534	
1535	       mknod /dev/tap0 c 36 0 + 16
1536	
1537	
1538	
1539	
1540	  You also need to make sure that the host kernel has ethertap support.
1541	  If ethertap is enabled as a module, you apparently need to insmod
1542	  ethertap once for each ethertap device you want to enable.  So,
1543	
1544	
1545	       host#
1546	       insmod ethertap
1547	
1548	
1549	
1550	
1551	  will give you the tap0 interface.  To get the tap1 interface, you need
1552	  to run
1553	
1554	
1555	       host#
1556	       insmod ethertap unit=1 -o ethertap1
1557	
1558	
1559	
1560	
1561	
1562	
1563	
1564	  6.9.  The switch daemon
1565	
1566	  Note: This is the daemon formerly known as uml_router, but which was
1567	  renamed so the network weenies of the world would stop growling at me.
1568	
1569	
1570	  The switch daemon, uml_switch, provides a mechanism for creating a
1571	  totally virtual network.  By default, it provides no connection to the
1572	  host network (but see -tap, below).
1573	
1574	
1575	  The first thing you need to do is run the daemon.  Running it with no
1576	  arguments will make it listen on a default pair of unix domain
1577	  sockets.
1578	
1579	
1580	  If you want it to listen on a different pair of sockets, use
1581	
1582	
1583	        -unix control socket data socket
1584	
1585	
1586	
1587	
1588	
1589	  If you want it to act as a hub rather than a switch, use
1590	
1591	
1592	        -hub
1593	
1594	
1595	
1596	
1597	
1598	  If you want the switch to be connected to host networking (allowing
1599	  the umls to get access to the outside world through the host), use
1600	
1601	
1602	        -tap tap0
1603	
1604	
1605	
1606	
1607	
1608	  Note that the tap device must be preconfigured (see "TUN/TAP with a
1609	  preconfigured tap device", above).  If you're using a different tap
1610	  device than tap0, specify that instead of tap0.
1611	
1612	
1613	  uml_switch can be backgrounded as follows
1614	
1615	
1616	       host%
1617	       uml_switch [ options ] < /dev/null > /dev/null
1618	
1619	
1620	
1621	
1622	  The reason it doesn't background by default is that it listens to
1623	  stdin for EOF.  When it sees that, it exits.
1624	
1625	
1626	  The general format of the kernel command line switch is
1627	
1628	
1629	
1630	       ethn=daemon,ethernet address,socket
1631	       type,control socket,data socket
1632	
1633	
1634	
1635	
1636	  You can leave off everything except the 'daemon'.  You only need to
1637	  specify the ethernet address if the one that will be assigned to it
1638	  isn't acceptable for some reason.  The rest of the arguments describe
1639	  how to communicate with the daemon.  You should only specify them if
1640	  you told the daemon to use different sockets than the default.  So, if
1641	  you ran the daemon with no arguments, running the UML on the same
1642	  machine with
1643	       eth0=daemon
1644	
1645	
1646	
1647	
1648	  will cause the eth0 driver to attach itself to the daemon correctly.
1649	
1650	
1651	
1652	  6.10.  Slip
1653	
1654	  Slip is another, less general, mechanism for a process to communicate
1655	  with the host networking.  In contrast to the ethertap interface,
1656	  which exchanges ethernet frames with the host and can be used to
1657	  transport any higher-level protocol, it can only be used to transport
1658	  IP.
1659	
1660	
1661	  The general format of the command line switch is
1662	
1663	
1664	
1665	       ethn=slip,slip IP
1666	
1667	
1668	
1669	
1670	  The slip IP argument is the IP address that will be assigned to the
1671	  host end of the slip device.  If it is specified, the helper will run
1672	  and will set up the host so that the virtual machine can reach it and
1673	  the rest of the network.
1674	
1675	
1676	  There are some oddities with this interface that you should be aware
1677	  of.  You should only specify one slip device on a given virtual
1678	  machine, and its name inside UML will be 'umn', not 'eth0' or whatever
1679	  you specified on the command line.  These problems will be fixed at
1680	  some point.
1681	
1682	
1683	
1684	  6.11.  Slirp
1685	
1686	  slirp uses an external program, usually /usr/bin/slirp, to provide IP
1687	  only networking connectivity through the host. This is similar to IP
1688	  masquerading with a firewall, although the translation is performed in
1689	  user-space, rather than by the kernel.  As slirp does not set up any
1690	  interfaces on the host, or changes routing, slirp does not require
1691	  root access or setuid binaries on the host.
1692	
1693	
1694	  The general format of the command line switch for slirp is:
1695	
1696	
1697	
1698	       ethn=slirp,ethernet address,slirp path
1699	
1700	
1701	
1702	
1703	  The ethernet address is optional, as UML will set up the interface
1704	  with an ethernet address based upon the initial IP address of the
1705	  interface.  The slirp path is generally /usr/bin/slirp, although it
1706	  will depend on distribution.
1707	
1708	
1709	  The slirp program can have a number of options passed to the command
1710	  line and we can't add them to the UML command line, as they will be
1711	  parsed incorrectly.  Instead, a wrapper shell script can be written or
1712	  the options inserted into the  /.slirprc file.  More information on
1713	  all of the slirp options can be found in its man pages.
1714	
1715	
1716	  The eth0 interface on UML should be set up with the IP 10.2.0.15,
1717	  although you can use anything as long as it is not used by a network
1718	  you will be connecting to. The default route on UML should be set to
1719	  use
1720	
1721	
1722	       UML#
1723	       route add default dev eth0
1724	
1725	
1726	
1727	
1728	  slirp provides a number of useful IP addresses which can be used by
1729	  UML, such as 10.0.2.3 which is an alias for the DNS server specified
1730	  in /etc/resolv.conf on the host or the IP given in the 'dns' option
1731	  for slirp.
1732	
1733	
1734	  Even with a baudrate setting higher than 115200, the slirp connection
1735	  is limited to 115200. If you need it to go faster, the slirp binary
1736	  needs to be compiled with FULL_BOLT defined in config.h.
1737	
1738	
1739	
1740	  6.12.  pcap
1741	
1742	  The pcap transport is attached to a UML ethernet device on the command
1743	  line or with uml_mconsole with the following syntax:
1744	
1745	
1746	
1747	       ethn=pcap,host interface,filter
1748	       expression,option1,option2
1749	
1750	
1751	
1752	
1753	  The expression and options are optional.
1754	
1755	
1756	  The interface is whatever network device on the host you want to
1757	  sniff.  The expression is a pcap filter expression, which is also what
1758	  tcpdump uses, so if you know how to specify tcpdump filters, you will
1759	  use the same expressions here.  The options are up to two of
1760	  'promisc', control whether pcap puts the host interface into
1761	  promiscuous mode. 'optimize' and 'nooptimize' control whether the pcap
1762	  expression optimizer is used.
1763	
1764	
1765	  Example:
1766	
1767	
1768	
1769	       eth0=pcap,eth0,tcp
1770	
1771	       eth1=pcap,eth0,!tcp
1772	
1773	
1774	
1775	  will cause the UML eth0 to emit all tcp packets on the host eth0 and
1776	  the UML eth1 to emit all non-tcp packets on the host eth0.
1777	
1778	
1779	
1780	  6.13.  Setting up the host yourself
1781	
1782	  If you don't specify an address for the host side of the ethertap or
1783	  slip device, UML won't do any setup on the host.  So this is what is
1784	  needed to get things working (the examples use a host-side IP of
1785	  192.168.0.251 and a UML-side IP of 192.168.0.250 - adjust to suit your
1786	  own network):
1787	
1788	  o  The device needs to be configured with its IP address.  Tap devices
1789	     are also configured with an mtu of 1484.  Slip devices are
1790	     configured with a point-to-point address pointing at the UML ip
1791	     address.
1792	
1793	
1794	       host#  ifconfig tap0 arp mtu 1484 192.168.0.251 up
1795	
1796	
1797	
1798	
1799	
1800	
1801	       host#
1802	       ifconfig sl0 192.168.0.251 pointopoint 192.168.0.250 up
1803	
1804	
1805	
1806	
1807	
1808	  o  If a tap device is being set up, a route is set to the UML IP.
1809	
1810	
1811	       UML# route add -host 192.168.0.250 gw 192.168.0.251
1812	
1813	
1814	
1815	
1816	
1817	  o  To allow other hosts on your network to see the virtual machine,
1818	     proxy arp is set up for it.
1819	
1820	
1821	       host#  arp -Ds 192.168.0.250 eth0 pub
1822	
1823	
1824	
1825	
1826	
1827	  o  Finally, the host is set up to route packets.
1828	
1829	
1830	       host#  echo 1 > /proc/sys/net/ipv4/ip_forward
1831	
1832	
1833	
1834	
1835	
1836	
1837	
1838	
1839	
1840	
1841	  7.  Sharing Filesystems between Virtual Machines
1842	
1843	
1844	
1845	
1846	  7.1.  A warning
1847	
1848	  Don't attempt to share filesystems simply by booting two UMLs from the
1849	  same file.  That's the same thing as booting two physical machines
1850	  from a shared disk.  It will result in filesystem corruption.
1851	
1852	
1853	
1854	  7.2.  Using layered block devices
1855	
1856	  The way to share a filesystem between two virtual machines is to use
1857	  the copy-on-write (COW) layering capability of the ubd block driver.
1858	  As of 2.4.6-2um, the driver supports layering a read-write private
1859	  device over a read-only shared device.  A machine's writes are stored
1860	  in the private device, while reads come from either device - the
1861	  private one if the requested block is valid in it, the shared one if
1862	  not.  Using this scheme, the majority of data which is unchanged is
1863	  shared between an arbitrary number of virtual machines, each of which
1864	  has a much smaller file containing the changes that it has made.  With
1865	  a large number of UMLs booting from a large root filesystem, this
1866	  leads to a huge disk space saving.  It will also help performance,
1867	  since the host will be able to cache the shared data using a much
1868	  smaller amount of memory, so UML disk requests will be served from the
1869	  host's memory rather than its disks.
1870	
1871	
1872	
1873	
1874	  To add a copy-on-write layer to an existing block device file, simply
1875	  add the name of the COW file to the appropriate ubd switch:
1876	
1877	
1878	        ubd0=root_fs_cow,root_fs_debian_22
1879	
1880	
1881	
1882	
1883	  where 'root_fs_cow' is the private COW file and 'root_fs_debian_22' is
1884	  the existing shared filesystem.  The COW file need not exist.  If it
1885	  doesn't, the driver will create and initialize it.  Once the COW file
1886	  has been initialized, it can be used on its own on the command line:
1887	
1888	
1889	        ubd0=root_fs_cow
1890	
1891	
1892	
1893	
1894	  The name of the backing file is stored in the COW file header, so it
1895	  would be redundant to continue specifying it on the command line.
1896	
1897	
1898	
1899	  7.3.  Note!
1900	
1901	  When checking the size of the COW file in order to see the gobs of
1902	  space that you're saving, make sure you use 'ls -ls' to see the actual
1903	  disk consumption rather than the length of the file.  The COW file is
1904	  sparse, so the length will be very different from the disk usage.
1905	  Here is a 'ls -l' of a COW file and backing file from one boot and
1906	  shutdown:
1907	       host% ls -l cow.debian debian2.2
1908	       -rw-r--r--    1 jdike    jdike    492504064 Aug  6 21:16 cow.debian
1909	       -rwxrw-rw-    1 jdike    jdike    537919488 Aug  6 20:42 debian2.2
1910	
1911	
1912	
1913	
1914	  Doesn't look like much saved space, does it?  Well, here's 'ls -ls':
1915	
1916	
1917	       host% ls -ls cow.debian debian2.2
1918	          880 -rw-r--r--    1 jdike    jdike    492504064 Aug  6 21:16 cow.debian
1919	       525832 -rwxrw-rw-    1 jdike    jdike    537919488 Aug  6 20:42 debian2.2
1920	
1921	
1922	
1923	
1924	  Now, you can see that the COW file has less than a meg of disk, rather
1925	  than 492 meg.
1926	
1927	
1928	
1929	  7.4.  Another warning
1930	
1931	  Once a filesystem is being used as a readonly backing file for a COW
1932	  file, do not boot directly from it or modify it in any way.  Doing so
1933	  will invalidate any COW files that are using it.  The mtime and size
1934	  of the backing file are stored in the COW file header at its creation,
1935	  and they must continue to match.  If they don't, the driver will
1936	  refuse to use the COW file.
1937	
1938	
1939	
1940	
1941	  If you attempt to evade this restriction by changing either the
1942	  backing file or the COW header by hand, you will get a corrupted
1943	  filesystem.
1944	
1945	
1946	
1947	
1948	  Among other things, this means that upgrading the distribution in a
1949	  backing file and expecting that all of the COW files using it will see
1950	  the upgrade will not work.
1951	
1952	
1953	
1954	
1955	  7.5.  uml_moo : Merging a COW file with its backing file
1956	
1957	  Depending on how you use UML and COW devices, it may be advisable to
1958	  merge the changes in the COW file into the backing file every once in
1959	  a while.
1960	
1961	
1962	
1963	
1964	  The utility that does this is uml_moo.  Its usage is
1965	
1966	
1967	       host% uml_moo COW file new backing file
1968	
1969	
1970	
1971	
1972	  There's no need to specify the backing file since that information is
1973	  already in the COW file header.  If you're paranoid, boot the new
1974	  merged file, and if you're happy with it, move it over the old backing
1975	  file.
1976	
1977	
1978	
1979	
1980	  uml_moo creates a new backing file by default as a safety measure.  It
1981	  also has a destructive merge option which will merge the COW file
1982	  directly into its current backing file.  This is really only usable
1983	  when the backing file only has one COW file associated with it.  If
1984	  there are multiple COWs associated with a backing file, a -d merge of
1985	  one of them will invalidate all of the others.  However, it is
1986	  convenient if you're short of disk space, and it should also be
1987	  noticeably faster than a non-destructive merge.
1988	
1989	
1990	
1991	
1992	  uml_moo is installed with the UML deb and RPM.  If you didn't install
1993	  UML from one of those packages, you can also get it from the UML
1994	  utilities <http://user-mode-linux.sourceforge.net/
1995	  utilities>  tar file in tools/moo.
1996	
1997	
1998	
1999	
2000	
2001	
2002	
2003	
2004	  8.  Creating filesystems
2005	
2006	
2007	  You may want to create and mount new UML filesystems, either because
2008	  your root filesystem isn't large enough or because you want to use a
2009	  filesystem other than ext2.
2010	
2011	
2012	  This was written on the occasion of reiserfs being included in the
2013	  2.4.1 kernel pool, and therefore the 2.4.1 UML, so the examples will
2014	  talk about reiserfs.  This information is generic, and the examples
2015	  should be easy to translate to the filesystem of your choice.
2016	
2017	
2018	  8.1.  Create the filesystem file
2019	
2020	  dd is your friend.  All you need to do is tell dd to create an empty
2021	  file of the appropriate size.  I usually make it sparse to save time
2022	  and to avoid allocating disk space until it's actually used.  For
2023	  example, the following command will create a sparse 100 meg file full
2024	  of zeroes.
2025	
2026	
2027	       host%
2028	       dd if=/dev/zero of=new_filesystem seek=100 count=1 bs=1M
2029	
2030	
2031	
2032	
2033	
2034	
2035	  8.2.  Assign the file to a UML device
2036	
2037	  Add an argument like the following to the UML command line:
2038	
2039	  ubd4=new_filesystem
2040	
2041	
2042	
2043	
2044	  making sure that you use an unassigned ubd device number.
2045	
2046	
2047	
2048	  8.3.  Creating and mounting the filesystem
2049	
2050	  Make sure that the filesystem is available, either by being built into
2051	  the kernel, or available as a module, then boot up UML and log in.  If
2052	  the root filesystem doesn't have the filesystem utilities (mkfs, fsck,
2053	  etc), then get them into UML by way of the net or hostfs.
2054	
2055	
2056	  Make the new filesystem on the device assigned to the new file:
2057	
2058	
2059	       host#  mkreiserfs /dev/ubd/4
2060	
2061	
2062	       <----------- MKREISERFSv2 ----------->
2063	
2064	       ReiserFS version 3.6.25
2065	       Block size 4096 bytes
2066	       Block count 25856
2067	       Used blocks 8212
2068	               Journal - 8192 blocks (18-8209), journal header is in block 8210
2069	               Bitmaps: 17
2070	               Root block 8211
2071	       Hash function "r5"
2072	       ATTENTION: ALL DATA WILL BE LOST ON '/dev/ubd/4'! (y/n)y
2073	       journal size 8192 (from 18)
2074	       Initializing journal - 0%....20%....40%....60%....80%....100%
2075	       Syncing..done.
2076	
2077	
2078	
2079	
2080	  Now, mount it:
2081	
2082	
2083	       UML#
2084	       mount /dev/ubd/4 /mnt
2085	
2086	
2087	
2088	
2089	  and you're in business.
2090	
2091	
2092	
2093	
2094	
2095	
2096	
2097	
2098	
2099	  9.  Host file access
2100	
2101	
2102	  If you want to access files on the host machine from inside UML, you
2103	  can treat it as a separate machine and either nfs mount directories
2104	  from the host or copy files into the virtual machine with scp or rcp.
2105	  However, since UML is running on the host, it can access those
2106	  files just like any other process and make them available inside the
2107	  virtual machine without needing to use the network.
2108	
2109	
2110	  This is now possible with the hostfs virtual filesystem.  With it, you
2111	  can mount a host directory into the UML filesystem and access the
2112	  files contained in it just as you would on the host.
2113	
2114	
2115	  9.1.  Using hostfs
2116	
2117	  To begin with, make sure that hostfs is available inside the virtual
2118	  machine with
2119	
2120	
2121	       UML# cat /proc/filesystems
2122	
2123	
2124	
2125	  .  hostfs should be listed.  If it's not, either rebuild the kernel
2126	  with hostfs configured into it or make sure that hostfs is built as a
2127	  module and available inside the virtual machine, and insmod it.
2128	
2129	
2130	  Now all you need to do is run mount:
2131	
2132	
2133	       UML# mount none /mnt/host -t hostfs
2134	
2135	
2136	
2137	
2138	  will mount the host's / on the virtual machine's /mnt/host.
2139	
2140	
2141	  If you don't want to mount the host root directory, then you can
2142	  specify a subdirectory to mount with the -o switch to mount:
2143	
2144	
2145	       UML# mount none /mnt/home -t hostfs -o /home
2146	
2147	
2148	
2149	
2150	  will mount the hosts's /home on the virtual machine's /mnt/home.
2151	
2152	
2153	
2154	  9.2.  hostfs as the root filesystem
2155	
2156	  It's possible to boot from a directory hierarchy on the host using
2157	  hostfs rather than using the standard filesystem in a file.
2158	
2159	  To start, you need that hierarchy.  The easiest way is to loop mount
2160	  an existing root_fs file:
2161	
2162	
2163	       host#  mount root_fs uml_root_dir -o loop
2164	
2165	
2166	
2167	
2168	  You need to change the filesystem type of / in etc/fstab to be
2169	  'hostfs', so that line looks like this:
2170	
2171	  /dev/ubd/0       /        hostfs      defaults          1   1
2172	
2173	
2174	
2175	
2176	  Then you need to chown to yourself all the files in that directory
2177	  that are owned by root.  This worked for me:
2178	
2179	
2180	       host#  find . -uid 0 -exec chown jdike {} \;
2181	
2182	
2183	
2184	
2185	  Next, make sure that your UML kernel has hostfs compiled in, not as a
2186	  module.  Then run UML with the boot device pointing at that directory:
2187	
2188	
2189	        ubd0=/path/to/uml/root/directory
2190	
2191	
2192	
2193	
2194	  UML should then boot as it does normally.
2195	
2196	
2197	  9.3.  Building hostfs
2198	
2199	  If you need to build hostfs because it's not in your kernel, you have
2200	  two choices:
2201	
2202	
2203	
2204	  o  Compiling hostfs into the kernel:
2205	
2206	
2207	     Reconfigure the kernel and set the 'Host filesystem' option under
2208	
2209	
2210	  o  Compiling hostfs as a module:
2211	
2212	
2213	     Reconfigure the kernel and set the 'Host filesystem' option under
2214	     be in arch/um/fs/hostfs/hostfs.o.  Install that in
2215	     /lib/modules/`uname -r`/fs in the virtual machine, boot it up, and
2216	
2217	
2218	       UML# insmod hostfs
2219	
2220	
2221	
2222	
2223	
2224	
2225	
2226	
2227	
2228	
2229	
2230	
2231	  10.  The Management Console
2232	
2233	
2234	
2235	  The UML management console is a low-level interface to the kernel,
2236	  somewhat like the i386 SysRq interface.  Since there is a full-blown
2237	  operating system under UML, there is much greater flexibility possible
2238	  than with the SysRq mechanism.
2239	
2240	
2241	  There are a number of things you can do with the mconsole interface:
2242	
2243	  o  get the kernel version
2244	
2245	  o  add and remove devices
2246	
2247	  o  halt or reboot the machine
2248	
2249	  o  Send SysRq commands
2250	
2251	  o  Pause and resume the UML
2252	
2253	
2254	  You need the mconsole client (uml_mconsole) which is present in CVS
2255	  (/tools/mconsole) in 2.4.5-9um and later, and will be in the RPM in
2256	  2.4.6.
2257	
2258	
2259	  You also need CONFIG_MCONSOLE (under 'General Setup') enabled in UML.
2260	  When you boot UML, you'll see a line like:
2261	
2262	
2263	       mconsole initialized on /home/jdike/.uml/umlNJ32yL/mconsole
2264	
2265	
2266	
2267	
2268	  If you specify a unique machine id one the UML command line, i.e.
2269	
2270	
2271	        umid=debian
2272	
2273	
2274	
2275	
2276	  you'll see this
2277	
2278	
2279	       mconsole initialized on /home/jdike/.uml/debian/mconsole
2280	
2281	
2282	
2283	
2284	  That file is the socket that uml_mconsole will use to communicate with
2285	  UML.  Run it with either the umid or the full path as its argument:
2286	
2287	
2288	       host% uml_mconsole debian
2289	
2290	
2291	
2292	
2293	  or
2294	
2295	
2296	       host% uml_mconsole /home/jdike/.uml/debian/mconsole
2297	
2298	
2299	
2300	
2301	  You'll get a prompt, at which you can run one of these commands:
2302	
2303	  o  version
2304	
2305	  o  halt
2306	
2307	  o  reboot
2308	
2309	  o  config
2310	
2311	  o  remove
2312	
2313	  o  sysrq
2314	
2315	  o  help
2316	
2317	  o  cad
2318	
2319	  o  stop
2320	
2321	  o  go
2322	
2323	
2324	  10.1.  version
2325	
2326	  This takes no arguments.  It prints the UML version.
2327	
2328	
2329	       (mconsole)  version
2330	       OK Linux usermode 2.4.5-9um #1 Wed Jun 20 22:47:08 EDT 2001 i686
2331	
2332	
2333	
2334	
2335	  There are a couple actual uses for this.  It's a simple no-op which
2336	  can be used to check that a UML is running.  It's also a way of
2337	  sending an interrupt to the UML.  This is sometimes useful on SMP
2338	  hosts, where there's a bug which causes signals to UML to be lost,
2339	  often causing it to appear to hang.  Sending such a UML the mconsole
2340	  version command is a good way to 'wake it up' before networking has
2341	  been enabled, as it does not do anything to the function of the UML.
2342	
2343	
2344	
2345	  10.2.  halt and reboot
2346	
2347	  These take no arguments.  They shut the machine down immediately, with
2348	  no syncing of disks and no clean shutdown of userspace.  So, they are
2349	  pretty close to crashing the machine.
2350	
2351	
2352	       (mconsole)  halt
2353	       OK
2354	
2355	
2356	
2357	
2358	
2359	
2360	  10.3.  config
2361	
2362	  "config" adds a new device to the virtual machine.  Currently the ubd
2363	  and network drivers support this.  It takes one argument, which is the
2364	  device to add, with the same syntax as the kernel command line.
2365	
2366	
2367	
2368	
2369	  (mconsole)
2370	  config ubd3=/home/jdike/incoming/roots/root_fs_debian22
2371	
2372	  OK
2373	  (mconsole)  config eth1=mcast
2374	  OK
2375	
2376	
2377	
2378	
2379	
2380	
2381	  10.4.  remove
2382	
2383	  "remove" deletes a device from the system.  Its argument is just the
2384	  name of the device to be removed. The device must be idle in whatever
2385	  sense the driver considers necessary.  In the case of the ubd driver,
2386	  the removed block device must not be mounted, swapped on, or otherwise
2387	  open, and in the case of the network driver, the device must be down.
2388	
2389	
2390	       (mconsole)  remove ubd3
2391	       OK
2392	       (mconsole)  remove eth1
2393	       OK
2394	
2395	
2396	
2397	
2398	
2399	
2400	  10.5.  sysrq
2401	
2402	  This takes one argument, which is a single letter.  It calls the
2403	  generic kernel's SysRq driver, which does whatever is called for by
2404	  that argument.  See the SysRq documentation in
2405	  Documentation/admin-guide/sysrq.rst in your favorite kernel tree to
2406	  see what letters are valid and what they do.
2407	
2408	
2409	
2410	  10.6.  help
2411	
2412	  "help" returns a string listing the valid commands and what each one
2413	  does.
2414	
2415	
2416	
2417	  10.7.  cad
2418	
2419	  This invokes the Ctl-Alt-Del action on init.  What exactly this ends
2420	  up doing is up to /etc/inittab.  Normally, it reboots the machine.
2421	  With UML, this is usually not desired, so if a halt would be better,
2422	  then find the section of inittab that looks like this
2423	
2424	
2425	       # What to do when CTRL-ALT-DEL is pressed.
2426	       ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now
2427	
2428	
2429	
2430	
2431	  and change the command to halt.
2432	
2433	
2434	
2435	  10.8.  stop
2436	
2437	  This puts the UML in a loop reading mconsole requests until a 'go'
2438	  mconsole command is received. This is very useful for making backups
2439	  of UML filesystems, as the UML can be stopped, then synced via 'sysrq
2440	  s', so that everything is written to the filesystem. You can then copy
2441	  the filesystem and then send the UML 'go' via mconsole.
2442	
2443	
2444	  Note that a UML running with more than one CPU will have problems
2445	  after you send the 'stop' command, as only one CPU will be held in a
2446	  mconsole loop and all others will continue as normal.  This is a bug,
2447	  and will be fixed.
2448	
2449	
2450	
2451	  10.9.  go
2452	
2453	  This resumes a UML after being paused by a 'stop' command. Note that
2454	  when the UML has resumed, TCP connections may have timed out and if
2455	  the UML is paused for a long period of time, crond might go a little
2456	  crazy, running all the jobs it didn't do earlier.
2457	
2458	
2459	
2460	
2461	
2462	
2463	
2464	
2465	  11.  Kernel debugging
2466	
2467	
2468	  Note: The interface that makes debugging, as described here, possible
2469	  is present in 2.4.0-test6 kernels and later.
2470	
2471	
2472	  Since the user-mode kernel runs as a normal Linux process, it is
2473	  possible to debug it with gdb almost like any other process.  It is
2474	  slightly different because the kernel's threads are already being
2475	  ptraced for system call interception, so gdb can't ptrace them.
2476	  However, a mechanism has been added to work around that problem.
2477	
2478	
2479	  In order to debug the kernel, you need build it from source.  See
2480	  ``Compiling the kernel and modules''  for information on doing that.
2481	  Make sure that you enable CONFIG_DEBUGSYM and CONFIG_PT_PROXY during
2482	  the config.  These will compile the kernel with -g, and enable the
2483	  ptrace proxy so that gdb works with UML, respectively.
2484	
2485	
2486	
2487	
2488	  11.1.  Starting the kernel under gdb
2489	
2490	  You can have the kernel running under the control of gdb from the
2491	  beginning by putting 'debug' on the command line.  You will get an
2492	  xterm with gdb running inside it.  The kernel will send some commands
2493	  to gdb which will leave it stopped at the beginning of start_kernel.
2494	  At this point, you can get things going with 'next', 'step', or
2495	  'cont'.
2496	
2497	
2498	  There is a transcript of a debugging session  here <debug-
2499	  session.html> , with breakpoints being set in the scheduler and in an
2500	  interrupt handler.
2501	  11.2.  Examining sleeping processes
2502	
2503	  Not every bug is evident in the currently running process.  Sometimes,
2504	  processes hang in the kernel when they shouldn't because they've
2505	  deadlocked on a semaphore or something similar.  In this case, when
2506	  you ^C gdb and get a backtrace, you will see the idle thread, which
2507	  isn't very relevant.
2508	
2509	
2510	  What you want is the stack of whatever process is sleeping when it
2511	  shouldn't be.  You need to figure out which process that is, which is
2512	  generally fairly easy.  Then you need to get its host process id,
2513	  which you can do either by looking at ps on the host or at
2514	  task.thread.extern_pid in gdb.
2515	
2516	
2517	  Now what you do is this:
2518	
2519	  o  detach from the current thread
2520	
2521	
2522	       (UML gdb)  det
2523	
2524	
2525	
2526	
2527	
2528	  o  attach to the thread you are interested in
2529	
2530	
2531	       (UML gdb)  att <host pid>
2532	
2533	
2534	
2535	
2536	
2537	  o  look at its stack and anything else of interest
2538	
2539	
2540	       (UML gdb)  bt
2541	
2542	
2543	
2544	
2545	  Note that you can't do anything at this point that requires that a
2546	  process execute, e.g. calling a function
2547	
2548	  o  when you're done looking at that process, reattach to the current
2549	     thread and continue it
2550	
2551	
2552	       (UML gdb)
2553	       att 1
2554	
2555	
2556	
2557	
2558	
2559	
2560	       (UML gdb)
2561	       c
2562	
2563	
2564	
2565	
2566	  Here, specifying any pid which is not the process id of a UML thread
2567	  will cause gdb to reattach to the current thread.  I commonly use 1,
2568	  but any other invalid pid would work.
2569	
2570	
2571	
2572	  11.3.  Running ddd on UML
2573	
2574	  ddd works on UML, but requires a special kludge.  The process goes
2575	  like this:
2576	
2577	  o  Start ddd
2578	
2579	
2580	       host% ddd linux
2581	
2582	
2583	
2584	
2585	
2586	  o  With ps, get the pid of the gdb that ddd started.  You can ask the
2587	     gdb to tell you, but for some reason that confuses things and
2588	     causes a hang.
2589	
2590	  o  run UML with 'debug=parent gdb-pid=<pid>' added to the command line
2591	     - it will just sit there after you hit return
2592	
2593	  o  type 'att 1' to the ddd gdb and you will see something like
2594	
2595	
2596	       0xa013dc51 in __kill ()
2597	
2598	
2599	       (gdb)
2600	
2601	
2602	
2603	
2604	
2605	  o  At this point, type 'c', UML will boot up, and you can use ddd just
2606	     as you do on any other process.
2607	
2608	
2609	
2610	  11.4.  Debugging modules
2611	
2612	  gdb has support for debugging code which is dynamically loaded into
2613	  the process.  This support is what is needed to debug kernel modules
2614	  under UML.
2615	
2616	
2617	  Using that support is somewhat complicated.  You have to tell gdb what
2618	  object file you just loaded into UML and where in memory it is.  Then,
2619	  it can read the symbol table, and figure out where all the symbols are
2620	  from the load address that you provided.  It gets more interesting
2621	  when you load the module again (i.e. after an rmmod).  You have to
2622	  tell gdb to forget about all its symbols, including the main UML ones
2623	  for some reason, then load then all back in again.
2624	
2625	
2626	  There's an easy way and a hard way to do this.  The easy way is to use
2627	  the umlgdb expect script written by Chandan Kudige.  It basically
2628	  automates the process for you.
2629	
2630	
2631	  First, you must tell it where your modules are.  There is a list in
2632	  the script that looks like this:
2633	       set MODULE_PATHS {
2634	       "fat" "/usr/src/uml/linux-2.4.18/fs/fat/fat.o"
2635	       "isofs" "/usr/src/uml/linux-2.4.18/fs/isofs/isofs.o"
2636	       "minix" "/usr/src/uml/linux-2.4.18/fs/minix/minix.o"
2637	       }
2638	
2639	
2640	
2641	
2642	  You change that to list the names and paths of the modules that you
2643	  are going to debug.  Then you run it from the toplevel directory of
2644	  your UML pool and it basically tells you what to do:
2645	
2646	
2647	
2648	
2649	                   ******** GDB pid is 21903 ********
2650	       Start UML as: ./linux <kernel switches> debug gdb-pid=21903
2651	
2652	
2653	
2654	       GNU gdb 5.0rh-5 Red Hat Linux 7.1
2655	       Copyright 2001 Free Software Foundation, Inc.
2656	       GDB is free software, covered by the GNU General Public License, and you are
2657	       welcome to change it and/or distribute copies of it under certain conditions.
2658	       Type "show copying" to see the conditions.
2659	       There is absolutely no warranty for GDB.  Type "show warranty" for details.
2660	       This GDB was configured as "i386-redhat-linux"...
2661	       (gdb) b sys_init_module
2662	       Breakpoint 1 at 0xa0011923: file module.c, line 349.
2663	       (gdb) att 1
2664	
2665	
2666	
2667	
2668	  After you run UML and it sits there doing nothing, you hit return at
2669	  the 'att 1' and continue it:
2670	
2671	
2672	       Attaching to program: /home/jdike/linux/2.4/um/./linux, process 1
2673	       0xa00f4221 in __kill ()
2674	       (UML gdb)  c
2675	       Continuing.
2676	
2677	
2678	
2679	
2680	  At this point, you debug normally.  When you insmod something, the
2681	  expect magic will kick in and you'll see something like:
2682	
2683	
2684	
2685	
2686	
2687	
2688	
2689	
2690	
2691	
2692	
2693	
2694	
2695	
2696	
2697	
2698	
2699	   *** Module hostfs loaded ***
2700	  Breakpoint 1, sys_init_module (name_user=0x805abb0 "hostfs",
2701	      mod_user=0x8070e00) at module.c:349
2702	  349             char *name, *n_name, *name_tmp = NULL;
2703	  (UML gdb)  finish
2704	  Run till exit from #0  sys_init_module (name_user=0x805abb0 "hostfs",
2705	      mod_user=0x8070e00) at module.c:349
2706	  0xa00e2e23 in execute_syscall (r=0xa8140284) at syscall_kern.c:411
2707	  411             else res = EXECUTE_SYSCALL(syscall, regs);
2708	  Value returned is $1 = 0
2709	  (UML gdb)
2710	  p/x (int)module_list + module_list->size_of_struct
2711	
2712	  $2 = 0xa9021054
2713	  (UML gdb)  symbol-file ./linux
2714	  Load new symbol table from "./linux"? (y or n) y
2715	  Reading symbols from ./linux...
2716	  done.
2717	  (UML gdb)
2718	  add-symbol-file /home/jdike/linux/2.4/um/arch/um/fs/hostfs/hostfs.o 0xa9021054
2719	
2720	  add symbol table from file "/home/jdike/linux/2.4/um/arch/um/fs/hostfs/hostfs.o" at
2721	          .text_addr = 0xa9021054
2722	   (y or n) y
2723	
2724	  Reading symbols from /home/jdike/linux/2.4/um/arch/um/fs/hostfs/hostfs.o...
2725	  done.
2726	  (UML gdb)  p *module_list
2727	  $1 = {size_of_struct = 84, next = 0xa0178720, name = 0xa9022de0 "hostfs",
2728	    size = 9016, uc = {usecount = {counter = 0}, pad = 0}, flags = 1,
2729	    nsyms = 57, ndeps = 0, syms = 0xa9023170, deps = 0x0, refs = 0x0,
2730	    init = 0xa90221f0 <init_hostfs>, cleanup = 0xa902222c <exit_hostfs>,
2731	    ex_table_start = 0x0, ex_table_end = 0x0, persist_start = 0x0,
2732	    persist_end = 0x0, can_unload = 0, runsize = 0, kallsyms_start = 0x0,
2733	    kallsyms_end = 0x0,
2734	    archdata_start = 0x1b855 <Address 0x1b855 out of bounds>,
2735	    archdata_end = 0xe5890000 <Address 0xe5890000 out of bounds>,
2736	    kernel_data = 0xf689c35d <Address 0xf689c35d out of bounds>}
2737	  >> Finished loading symbols for hostfs ...
2738	
2739	
2740	
2741	
2742	  That's the easy way.  It's highly recommended.  The hard way is
2743	  described below in case you're interested in what's going on.
2744	
2745	
2746	  Boot the kernel under the debugger and load the module with insmod or
2747	  modprobe.  With gdb, do:
2748	
2749	
2750	       (UML gdb)  p module_list
2751	
2752	
2753	
2754	
2755	  This is a list of modules that have been loaded into the kernel, with
2756	  the most recently loaded module first.  Normally, the module you want
2757	  is at module_list.  If it's not, walk down the next links, looking at
2758	  the name fields until find the module you want to debug.  Take the
2759	  address of that structure, and add module.size_of_struct (which in
2760	  2.4.10 kernels is 96 (0x60)) to it.  Gdb can make this hard addition
2761	  for you :-):
2762	
2763	
2764	
2765	  (UML gdb)
2766	  printf "%#x\n", (int)module_list module_list->size_of_struct
2767	
2768	
2769	
2770	
2771	  The offset from the module start occasionally changes (before 2.4.0,
2772	  it was module.size_of_struct + 4), so it's a good idea to check the
2773	  init and cleanup addresses once in a while, as describe below.  Now
2774	  do:
2775	
2776	
2777	       (UML gdb)
2778	       add-symbol-file /path/to/module/on/host that_address
2779	
2780	
2781	
2782	
2783	  Tell gdb you really want to do it, and you're in business.
2784	
2785	
2786	  If there's any doubt that you got the offset right, like breakpoints
2787	  appear not to work, or they're appearing in the wrong place, you can
2788	  check it by looking at the module structure.  The init and cleanup
2789	  fields should look like:
2790	
2791	
2792	       init = 0x588066b0 <init_hostfs>, cleanup = 0x588066c0 <exit_hostfs>
2793	
2794	
2795	
2796	
2797	  with no offsets on the symbol names.  If the names are right, but they
2798	  are offset, then the offset tells you how much you need to add to the
2799	  address you gave to add-symbol-file.
2800	
2801	
2802	  When you want to load in a new version of the module, you need to get
2803	  gdb to forget about the old one.  The only way I've found to do that
2804	  is to tell gdb to forget about all symbols that it knows about:
2805	
2806	
2807	       (UML gdb)  symbol-file
2808	
2809	
2810	
2811	
2812	  Then reload the symbols from the kernel binary:
2813	
2814	
2815	       (UML gdb)  symbol-file /path/to/kernel
2816	
2817	
2818	
2819	
2820	  and repeat the process above.  You'll also need to re-enable break-
2821	  points.  They were disabled when you dumped all the symbols because
2822	  gdb couldn't figure out where they should go.
2823	
2824	
2825	
2826	  11.5.  Attaching gdb to the kernel
2827	
2828	  If you don't have the kernel running under gdb, you can attach gdb to
2829	  it later by sending the tracing thread a SIGUSR1.  The first line of
2830	  the console output identifies its pid:
2831	       tracing thread pid = 20093
2832	
2833	
2834	
2835	
2836	  When you send it the signal:
2837	
2838	
2839	       host% kill -USR1 20093
2840	
2841	
2842	
2843	
2844	  you will get an xterm with gdb running in it.
2845	
2846	
2847	  If you have the mconsole compiled into UML, then the mconsole client
2848	  can be used to start gdb:
2849	
2850	
2851	       (mconsole)  (mconsole) config gdb=xterm
2852	
2853	
2854	
2855	
2856	  will fire up an xterm with gdb running in it.
2857	
2858	
2859	
2860	  11.6.  Using alternate debuggers
2861	
2862	  UML has support for attaching to an already running debugger rather
2863	  than starting gdb itself.  This is present in CVS as of 17 Apr 2001.
2864	  I sent it to Alan for inclusion in the ac tree, and it will be in my
2865	  2.4.4 release.
2866	
2867	
2868	  This is useful when gdb is a subprocess of some UI, such as emacs or
2869	  ddd.  It can also be used to run debuggers other than gdb on UML.
2870	  Below is an example of using strace as an alternate debugger.
2871	
2872	
2873	  To do this, you need to get the pid of the debugger and pass it in
2874	  with the
2875	
2876	
2877	  If you are using gdb under some UI, then tell it to 'att 1', and
2878	  you'll find yourself attached to UML.
2879	
2880	
2881	  If you are using something other than gdb as your debugger, then
2882	  you'll need to get it to do the equivalent of 'att 1' if it doesn't do
2883	  it automatically.
2884	
2885	
2886	  An example of an alternate debugger is strace.  You can strace the
2887	  actual kernel as follows:
2888	
2889	  o  Run the following in a shell
2890	
2891	
2892	       host%
2893	       sh -c 'echo pid=$$; echo -n hit return; read x; exec strace -p 1 -o strace.out'
2894	
2895	
2896	
2897	  o  Run UML with 'debug' and 'gdb-pid=<pid>' with the pid printed out
2898	     by the previous command
2899	
2900	  o  Hit return in the shell, and UML will start running, and strace
2901	     output will start accumulating in the output file.
2902	
2903	     Note that this is different from running
2904	
2905	
2906	       host% strace ./linux
2907	
2908	
2909	
2910	
2911	  That will strace only the main UML thread, the tracing thread, which
2912	  doesn't do any of the actual kernel work.  It just oversees the vir-
2913	  tual machine.  In contrast, using strace as described above will show
2914	  you the low-level activity of the virtual machine.
2915	
2916	
2917	
2918	
2919	
2920	  12.  Kernel debugging examples
2921	
2922	  12.1.  The case of the hung fsck
2923	
2924	  When booting up the kernel, fsck failed, and dropped me into a shell
2925	  to fix things up.  I ran fsck -y, which hung:
2926	
2927	
2928	
2929	
2930	
2931	
2932	
2933	
2934	
2935	
2936	
2937	
2938	
2939	
2940	
2941	
2942	
2943	
2944	
2945	
2946	
2947	
2948	
2949	
2950	
2951	
2952	
2953	
2954	
2955	
2956	
2957	
2958	
2959	
2960	
2961	
2962	
2963	  Setting hostname uml                    [ OK ]
2964	  Checking root filesystem
2965	  /dev/fhd0 was not cleanly unmounted, check forced.
2966	  Error reading block 86894 (Attempt to read block from filesystem resulted in short read) while reading indirect blocks of inode 19780.
2967	
2968	  /dev/fhd0: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
2969	          (i.e., without -a or -p options)
2970	  [ FAILED ]
2971	
2972	  *** An error occurred during the file system check.
2973	  *** Dropping you to a shell; the system will reboot
2974	  *** when you leave the shell.
2975	  Give root password for maintenance
2976	  (or type Control-D for normal startup):
2977	
2978	  [root@uml /root]# fsck -y /dev/fhd0
2979	  fsck -y /dev/fhd0
2980	  Parallelizing fsck version 1.14 (9-Jan-1999)
2981	  e2fsck 1.14, 9-Jan-1999 for EXT2 FS 0.5b, 95/08/09
2982	  /dev/fhd0 contains a file system with errors, check forced.
2983	  Pass 1: Checking inodes, blocks, and sizes
2984	  Error reading block 86894 (Attempt to read block from filesystem resulted in short read) while reading indirect blocks of inode 19780.  Ignore error? yes
2985	
2986	  Inode 19780, i_blocks is 1548, should be 540.  Fix? yes
2987	
2988	  Pass 2: Checking directory structure
2989	  Error reading block 49405 (Attempt to read block from filesystem resulted in short read).  Ignore error? yes
2990	
2991	  Directory inode 11858, block 0, offset 0: directory corrupted
2992	  Salvage? yes
2993	
2994	  Missing '.' in directory inode 11858.
2995	  Fix? yes
2996	
2997	  Missing '..' in directory inode 11858.
2998	  Fix? yes
2999	
3000	
3001	
3002	
3003	
3004	  The standard drill in this sort of situation is to fire up gdb on the
3005	  signal thread, which, in this case, was pid 1935.  In another window,
3006	  I run gdb and attach pid 1935.
3007	
3008	
3009	
3010	
3011	       ~/linux/2.3.26/um 1016: gdb linux
3012	       GNU gdb 4.17.0.11 with Linux support
3013	       Copyright 1998 Free Software Foundation, Inc.
3014	       GDB is free software, covered by the GNU General Public License, and you are
3015	       welcome to change it and/or distribute copies of it under certain conditions.
3016	       Type "show copying" to see the conditions.
3017	       There is absolutely no warranty for GDB.  Type "show warranty" for details.
3018	       This GDB was configured as "i386-redhat-linux"...
3019	
3020	       (gdb) att 1935
3021	       Attaching to program `/home/dike/linux/2.3.26/um/linux', Pid 1935
3022	       0x100756d9 in __wait4 ()
3023	
3024	
3025	
3026	
3027	
3028	
3029	  Let's see what's currently running:
3030	
3031	
3032	
3033	       (gdb) p current_task.pid
3034	       $1 = 0
3035	
3036	
3037	
3038	
3039	
3040	  It's the idle thread, which means that fsck went to sleep for some
3041	  reason and never woke up.
3042	
3043	
3044	  Let's guess that the last process in the process list is fsck:
3045	
3046	
3047	
3048	       (gdb) p current_task.prev_task.comm
3049	       $13 = "fsck.ext2\000\000\000\000\000\000"
3050	
3051	
3052	
3053	
3054	
3055	  It is, so let's see what it thinks it's up to:
3056	
3057	
3058	
3059	       (gdb) p current_task.prev_task.thread
3060	       $14 = {extern_pid = 1980, tracing = 0, want_tracing = 0, forking = 0,
3061	         kernel_stack_page = 0, signal_stack = 1342627840, syscall = {id = 4, args = {
3062	             3, 134973440, 1024, 0, 1024}, have_result = 0, result = 50590720},
3063	         request = {op = 2, u = {exec = {ip = 1350467584, sp = 2952789424}, fork = {
3064	               regs = {1350467584, 2952789424, 0 <repeats 15 times>}, sigstack = 0,
3065	               pid = 0}, switch_to = 0x507e8000, thread = {proc = 0x507e8000,
3066	               arg = 0xaffffdb0, flags = 0, new_pid = 0}, input_request = {
3067	               op = 1350467584, fd = -1342177872, proc = 0, pid = 0}}}}
3068	
3069	
3070	
3071	
3072	
3073	  The interesting things here are the fact that its .thread.syscall.id
3074	  is __NR_write (see the big switch in arch/um/kernel/syscall_kern.c or
3075	  the defines in include/asm-um/arch/unistd.h), and that it never
3076	  returned.  Also, its .request.op is OP_SWITCH (see
3077	  arch/um/include/user_util.h).  These mean that it went into a write,
3078	  and, for some reason, called schedule().
3079	
3080	
3081	  The fact that it never returned from write means that its stack should
3082	  be fairly interesting.  Its pid is 1980 (.thread.extern_pid).  That
3083	  process is being ptraced by the signal thread, so it must be detached
3084	  before gdb can attach it:
3085	
3086	
3087	
3088	
3089	
3090	
3091	
3092	
3093	
3094	
3095	  (gdb) call detach(1980)
3096	
3097	  Program received signal SIGSEGV, Segmentation fault.
3098	  <function called from gdb>
3099	  The program being debugged stopped while in a function called from GDB.
3100	  When the function (detach) is done executing, GDB will silently
3101	  stop (instead of continuing to evaluate the expression containing
3102	  the function call).
3103	  (gdb) call detach(1980)
3104	  $15 = 0
3105	
3106	
3107	
3108	
3109	
3110	  The first detach segfaults for some reason, and the second one
3111	  succeeds.
3112	
3113	
3114	  Now I detach from the signal thread, attach to the fsck thread, and
3115	  look at its stack:
3116	
3117	
3118	       (gdb) det
3119	       Detaching from program: /home/dike/linux/2.3.26/um/linux Pid 1935
3120	       (gdb) att 1980
3121	       Attaching to program `/home/dike/linux/2.3.26/um/linux', Pid 1980
3122	       0x10070451 in __kill ()
3123	       (gdb) bt
3124	       #0  0x10070451 in __kill ()
3125	       #1  0x10068ccd in usr1_pid (pid=1980) at process.c:30
3126	       #2  0x1006a03f in _switch_to (prev=0x50072000, next=0x507e8000)
3127	           at process_kern.c:156
3128	       #3  0x1006a052 in switch_to (prev=0x50072000, next=0x507e8000, last=0x50072000)
3129	           at process_kern.c:161
3130	       #4  0x10001d12 in schedule () at core.c:777
3131	       #5  0x1006a744 in __down (sem=0x507d241c) at semaphore.c:71
3132	       #6  0x1006aa10 in __down_failed () at semaphore.c:157
3133	       #7  0x1006c5d8 in segv_handler (sc=0x5006e940) at trap_user.c:174
3134	       #8  0x1006c5ec in kern_segv_handler (sig=11) at trap_user.c:182
3135	       #9  <signal handler called>
3136	       #10 0x10155404 in errno ()
3137	       #11 0x1006c0aa in segv (address=1342179328, is_write=2) at trap_kern.c:50
3138	       #12 0x1006c5d8 in segv_handler (sc=0x5006eaf8) at trap_user.c:174
3139	       #13 0x1006c5ec in kern_segv_handler (sig=11) at trap_user.c:182
3140	       #14 <signal handler called>
3141	       #15 0xc0fd in ?? ()
3142	       #16 0x10016647 in sys_write (fd=3,
3143	           buf=0x80b8800 <Address 0x80b8800 out of bounds>, count=1024)
3144	           at read_write.c:159
3145	       #17 0x1006d5b3 in execute_syscall (syscall=4, args=0x5006ef08)
3146	           at syscall_kern.c:254
3147	       #18 0x1006af87 in really_do_syscall (sig=12) at syscall_user.c:35
3148	       #19 <signal handler called>
3149	       #20 0x400dc8b0 in ?? ()
3150	
3151	
3152	
3153	
3154	
3155	  The interesting things here are :
3156	
3157	  o  There are two segfaults on this stack (frames 9 and 14)
3158	
3159	  o  The first faulting address (frame 11) is 0x50000800
3160	
3161	  (gdb) p (void *)1342179328
3162	  $16 = (void *) 0x50000800
3163	
3164	
3165	
3166	
3167	
3168	  The initial faulting address is interesting because it is on the idle
3169	  thread's stack.  I had been seeing the idle thread segfault for no
3170	  apparent reason, and the cause looked like stack corruption.  In hopes
3171	  of catching the culprit in the act, I had turned off all protections
3172	  to that stack while the idle thread wasn't running.  This apparently
3173	  tripped that trap.
3174	
3175	
3176	  However, the more immediate problem is that second segfault and I'm
3177	  going to concentrate on that.  First, I want to see where the fault
3178	  happened, so I have to go look at the sigcontent struct in frame 8:
3179	
3180	
3181	
3182	       (gdb) up
3183	       #1  0x10068ccd in usr1_pid (pid=1980) at process.c:30
3184	       30        kill(pid, SIGUSR1);
3185	       (gdb)
3186	       #2  0x1006a03f in _switch_to (prev=0x50072000, next=0x507e8000)
3187	           at process_kern.c:156
3188	       156       usr1_pid(getpid());
3189	       (gdb)
3190	       #3  0x1006a052 in switch_to (prev=0x50072000, next=0x507e8000, last=0x50072000)
3191	           at process_kern.c:161
3192	       161       _switch_to(prev, next);
3193	       (gdb)
3194	       #4  0x10001d12 in schedule () at core.c:777
3195	       777             switch_to(prev, next, prev);
3196	       (gdb)
3197	       #5  0x1006a744 in __down (sem=0x507d241c) at semaphore.c:71
3198	       71                      schedule();
3199	       (gdb)
3200	       #6  0x1006aa10 in __down_failed () at semaphore.c:157
3201	       157     }
3202	       (gdb)
3203	       #7  0x1006c5d8 in segv_handler (sc=0x5006e940) at trap_user.c:174
3204	       174       segv(sc->cr2, sc->err & 2);
3205	       (gdb)
3206	       #8  0x1006c5ec in kern_segv_handler (sig=11) at trap_user.c:182
3207	       182       segv_handler(sc);
3208	       (gdb) p *sc
3209	       Cannot access memory at address 0x0.
3210	
3211	
3212	
3213	
3214	  That's not very useful, so I'll try a more manual method:
3215	
3216	
3217	       (gdb) p *((struct sigcontext *) (&sig + 1))
3218	       $19 = {gs = 0, __gsh = 0, fs = 0, __fsh = 0, es = 43, __esh = 0, ds = 43,
3219	         __dsh = 0, edi = 1342179328, esi = 1350378548, ebp = 1342630440,
3220	         esp = 1342630420, ebx = 1348150624, edx = 1280, ecx = 0, eax = 0,
3221	         trapno = 14, err = 4, eip = 268480945, cs = 35, __csh = 0, eflags = 66118,
3222	         esp_at_signal = 1342630420, ss = 43, __ssh = 0, fpstate = 0x0, oldmask = 0,
3223	         cr2 = 1280}
3224	
3225	
3226	
3227	  The ip is in handle_mm_fault:
3228	
3229	
3230	       (gdb) p (void *)268480945
3231	       $20 = (void *) 0x1000b1b1
3232	       (gdb) i sym $20
3233	       handle_mm_fault + 57 in section .text
3234	
3235	
3236	
3237	
3238	
3239	  Specifically, it's in pte_alloc:
3240	
3241	
3242	       (gdb) i line *$20
3243	       Line 124 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
3244	          starts at address 0x1000b1b1 <handle_mm_fault+57>
3245	          and ends at 0x1000b1b7 <handle_mm_fault+63>.
3246	
3247	
3248	
3249	
3250	
3251	  To find where in handle_mm_fault this is, I'll jump forward in the
3252	  code until I see an address in that procedure:
3253	
3254	
3255	
3256	       (gdb) i line *0x1000b1c0
3257	       Line 126 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
3258	          starts at address 0x1000b1b7 <handle_mm_fault+63>
3259	          and ends at 0x1000b1c3 <handle_mm_fault+75>.
3260	       (gdb) i line *0x1000b1d0
3261	       Line 131 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
3262	          starts at address 0x1000b1d0 <handle_mm_fault+88>
3263	          and ends at 0x1000b1da <handle_mm_fault+98>.
3264	       (gdb) i line *0x1000b1e0
3265	       Line 61 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
3266	          starts at address 0x1000b1da <handle_mm_fault+98>
3267	          and ends at 0x1000b1e1 <handle_mm_fault+105>.
3268	       (gdb) i line *0x1000b1f0
3269	       Line 134 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
3270	          starts at address 0x1000b1f0 <handle_mm_fault+120>
3271	          and ends at 0x1000b200 <handle_mm_fault+136>.
3272	       (gdb) i line *0x1000b200
3273	       Line 135 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
3274	          starts at address 0x1000b200 <handle_mm_fault+136>
3275	          and ends at 0x1000b208 <handle_mm_fault+144>.
3276	       (gdb) i line *0x1000b210
3277	       Line 139 of "/home/dike/linux/2.3.26/um/include/asm/pgalloc.h"
3278	          starts at address 0x1000b210 <handle_mm_fault+152>
3279	          and ends at 0x1000b219 <handle_mm_fault+161>.
3280	       (gdb) i line *0x1000b220
3281	       Line 1168 of "memory.c" starts at address 0x1000b21e <handle_mm_fault+166>
3282	          and ends at 0x1000b222 <handle_mm_fault+170>.
3283	
3284	
3285	
3286	
3287	
3288	  Something is apparently wrong with the page tables or vma_structs, so
3289	  lets go back to frame 11 and have a look at them:
3290	
3291	
3292	
3293	  #11 0x1006c0aa in segv (address=1342179328, is_write=2) at trap_kern.c:50
3294	  50        handle_mm_fault(current, vma, address, is_write);
3295	  (gdb) call pgd_offset_proc(vma->vm_mm, address)
3296	  $22 = (pgd_t *) 0x80a548c
3297	
3298	
3299	
3300	
3301	
3302	  That's pretty bogus.  Page tables aren't supposed to be in process
3303	  text or data areas.  Let's see what's in the vma:
3304	
3305	
3306	       (gdb) p *vma
3307	       $23 = {vm_mm = 0x507d2434, vm_start = 0, vm_end = 134512640,
3308	         vm_next = 0x80a4f8c, vm_page_prot = {pgprot = 0}, vm_flags = 31200,
3309	         vm_avl_height = 2058, vm_avl_left = 0x80a8c94, vm_avl_right = 0x80d1000,
3310	         vm_next_share = 0xaffffdb0, vm_pprev_share = 0xaffffe63,
3311	         vm_ops = 0xaffffe7a, vm_pgoff = 2952789626, vm_file = 0xafffffec,
3312	         vm_private_data = 0x62}
3313	       (gdb) p *vma.vm_mm
3314	       $24 = {mmap = 0x507d2434, mmap_avl = 0x0, mmap_cache = 0x8048000,
3315	         pgd = 0x80a4f8c, mm_users = {counter = 0}, mm_count = {counter = 134904288},
3316	         map_count = 134909076, mmap_sem = {count = {counter = 135073792},
3317	           sleepers = -1342177872, wait = {lock = <optimized out or zero length>,
3318	             task_list = {next = 0xaffffe63, prev = 0xaffffe7a},
3319	             __magic = -1342177670, __creator = -1342177300}, __magic = 98},
3320	         page_table_lock = {}, context = 138, start_code = 0, end_code = 0,
3321	         start_data = 0, end_data = 0, start_brk = 0, brk = 0, start_stack = 0,
3322	         arg_start = 0, arg_end = 0, env_start = 0, env_end = 0, rss = 1350381536,
3323	         total_vm = 0, locked_vm = 0, def_flags = 0, cpu_vm_mask = 0, swap_cnt = 0,
3324	         swap_address = 0, segments = 0x0}
3325	
3326	
3327	
3328	
3329	
3330	  This also pretty bogus.  With all of the 0x80xxxxx and 0xaffffxxx
3331	  addresses, this is looking like a stack was plonked down on top of
3332	  these structures.  Maybe it's a stack overflow from the next page:
3333	
3334	
3335	
3336	       (gdb) p vma
3337	       $25 = (struct vm_area_struct *) 0x507d2434
3338	
3339	
3340	
3341	
3342	
3343	  That's towards the lower quarter of the page, so that would have to
3344	  have been pretty heavy stack overflow:
3345	
3346	
3347	
3348	
3349	
3350	
3351	
3352	
3353	
3354	
3355	
3356	
3357	
3358	
3359	  (gdb) x/100x $25
3360	  0x507d2434:     0x507d2434      0x00000000      0x08048000      0x080a4f8c
3361	  0x507d2444:     0x00000000      0x080a79e0      0x080a8c94      0x080d1000
3362	  0x507d2454:     0xaffffdb0      0xaffffe63      0xaffffe7a      0xaffffe7a
3363	  0x507d2464:     0xafffffec      0x00000062      0x0000008a      0x00000000
3364	  0x507d2474:     0x00000000      0x00000000      0x00000000      0x00000000
3365	  0x507d2484:     0x00000000      0x00000000      0x00000000      0x00000000
3366	  0x507d2494:     0x00000000      0x00000000      0x507d2fe0      0x00000000
3367	  0x507d24a4:     0x00000000      0x00000000      0x00000000      0x00000000
3368	  0x507d24b4:     0x00000000      0x00000000      0x00000000      0x00000000
3369	  0x507d24c4:     0x00000000      0x00000000      0x00000000      0x00000000
3370	  0x507d24d4:     0x00000000      0x00000000      0x00000000      0x00000000
3371	  0x507d24e4:     0x00000000      0x00000000      0x00000000      0x00000000
3372	  0x507d24f4:     0x00000000      0x00000000      0x00000000      0x00000000
3373	  0x507d2504:     0x00000000      0x00000000      0x00000000      0x00000000
3374	  0x507d2514:     0x00000000      0x00000000      0x00000000      0x00000000
3375	  0x507d2524:     0x00000000      0x00000000      0x00000000      0x00000000
3376	  0x507d2534:     0x00000000      0x00000000      0x507d25dc      0x00000000
3377	  0x507d2544:     0x00000000      0x00000000      0x00000000      0x00000000
3378	  0x507d2554:     0x00000000      0x00000000      0x00000000      0x00000000
3379	  0x507d2564:     0x00000000      0x00000000      0x00000000      0x00000000
3380	  0x507d2574:     0x00000000      0x00000000      0x00000000      0x00000000
3381	  0x507d2584:     0x00000000      0x00000000      0x00000000      0x00000000
3382	  0x507d2594:     0x00000000      0x00000000      0x00000000      0x00000000
3383	  0x507d25a4:     0x00000000      0x00000000      0x00000000      0x00000000
3384	  0x507d25b4:     0x00000000      0x00000000      0x00000000      0x00000000
3385	
3386	
3387	
3388	
3389	
3390	  It's not stack overflow.  The only "stack-like" piece of this data is
3391	  the vma_struct itself.
3392	
3393	
3394	  At this point, I don't see any avenues to pursue, so I just have to
3395	  admit that I have no idea what's going on.  What I will do, though, is
3396	  stick a trap on the segfault handler which will stop if it sees any
3397	  writes to the idle thread's stack.  That was the thing that happened
3398	  first, and it may be that if I can catch it immediately, what's going
3399	  on will be somewhat clearer.
3400	
3401	
3402	  12.2.  Episode 2: The case of the hung fsck
3403	
3404	  After setting a trap in the SEGV handler for accesses to the signal
3405	  thread's stack, I reran the kernel.
3406	
3407	
3408	  fsck hung again, this time by hitting the trap:
3409	
3410	
3411	
3412	
3413	
3414	
3415	
3416	
3417	
3418	
3419	
3420	
3421	
3422	
3423	
3424	
3425	  Setting hostname uml                            [ OK ]
3426	  Checking root filesystem
3427	  /dev/fhd0 contains a file system with errors, check forced.
3428	  Error reading block 86894 (Attempt to read block from filesystem resulted in short read) while reading indirect blocks of inode 19780.
3429	
3430	  /dev/fhd0: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY.
3431	          (i.e., without -a or -p options)
3432	  [ FAILED ]
3433	
3434	  *** An error occurred during the file system check.
3435	  *** Dropping you to a shell; the system will reboot
3436	  *** when you leave the shell.
3437	  Give root password for maintenance
3438	  (or type Control-D for normal startup):
3439	
3440	  [root@uml /root]# fsck -y /dev/fhd0
3441	  fsck -y /dev/fhd0
3442	  Parallelizing fsck version 1.14 (9-Jan-1999)
3443	  e2fsck 1.14, 9-Jan-1999 for EXT2 FS 0.5b, 95/08/09
3444	  /dev/fhd0 contains a file system with errors, check forced.
3445	  Pass 1: Checking inodes, blocks, and sizes
3446	  Error reading block 86894 (Attempt to read block from filesystem resulted in short read) while reading indirect blocks of inode 19780.  Ignore error? yes
3447	
3448	  Pass 2: Checking directory structure
3449	  Error reading block 49405 (Attempt to read block from filesystem resulted in short read).  Ignore error? yes
3450	
3451	  Directory inode 11858, block 0, offset 0: directory corrupted
3452	  Salvage? yes
3453	
3454	  Missing '.' in directory inode 11858.
3455	  Fix? yes
3456	
3457	  Missing '..' in directory inode 11858.
3458	  Fix? yes
3459	
3460	  Untested (4127) [100fe44c]: trap_kern.c line 31
3461	
3462	
3463	
3464	
3465	
3466	  I need to get the signal thread to detach from pid 4127 so that I can
3467	  attach to it with gdb.  This is done by sending it a SIGUSR1, which is
3468	  caught by the signal thread, which detaches the process:
3469	
3470	
3471	       kill -USR1 4127
3472	
3473	
3474	
3475	
3476	
3477	  Now I can run gdb on it:
3478	
3479	
3480	
3481	
3482	
3483	
3484	
3485	
3486	
3487	
3488	
3489	
3490	
3491	  ~/linux/2.3.26/um 1034: gdb linux
3492	  GNU gdb 4.17.0.11 with Linux support
3493	  Copyright 1998 Free Software Foundation, Inc.
3494	  GDB is free software, covered by the GNU General Public License, and you are
3495	  welcome to change it and/or distribute copies of it under certain conditions.
3496	  Type "show copying" to see the conditions.
3497	  There is absolutely no warranty for GDB.  Type "show warranty" for details.
3498	  This GDB was configured as "i386-redhat-linux"...
3499	  (gdb) att 4127
3500	  Attaching to program `/home/dike/linux/2.3.26/um/linux', Pid 4127
3501	  0x10075891 in __libc_nanosleep ()
3502	
3503	
3504	
3505	
3506	
3507	  The backtrace shows that it was in a write and that the fault address
3508	  (address in frame 3) is 0x50000800, which is right in the middle of
3509	  the signal thread's stack page:
3510	
3511	
3512	       (gdb) bt
3513	       #0  0x10075891 in __libc_nanosleep ()
3514	       #1  0x1007584d in __sleep (seconds=1000000)
3515	           at ../sysdeps/unix/sysv/linux/sleep.c:78
3516	       #2  0x1006ce9a in stop () at user_util.c:191
3517	       #3  0x1006bf88 in segv (address=1342179328, is_write=2) at trap_kern.c:31
3518	       #4  0x1006c628 in segv_handler (sc=0x5006eaf8) at trap_user.c:174
3519	       #5  0x1006c63c in kern_segv_handler (sig=11) at trap_user.c:182
3520	       #6  <signal handler called>
3521	       #7  0xc0fd in ?? ()
3522	       #8  0x10016647 in sys_write (fd=3, buf=0x80b8800 "R.", count=1024)
3523	           at read_write.c:159
3524	       #9  0x1006d603 in execute_syscall (syscall=4, args=0x5006ef08)
3525	           at syscall_kern.c:254
3526	       #10 0x1006af87 in really_do_syscall (sig=12) at syscall_user.c:35
3527	       #11 <signal handler called>
3528	       #12 0x400dc8b0 in ?? ()
3529	       #13 <signal handler called>
3530	       #14 0x400dc8b0 in ?? ()
3531	       #15 0x80545fd in ?? ()
3532	       #16 0x804daae in ?? ()
3533	       #17 0x8054334 in ?? ()
3534	       #18 0x804d23e in ?? ()
3535	       #19 0x8049632 in ?? ()
3536	       #20 0x80491d2 in ?? ()
3537	       #21 0x80596b5 in ?? ()
3538	       (gdb) p (void *)1342179328
3539	       $3 = (void *) 0x50000800
3540	
3541	
3542	
3543	
3544	
3545	  Going up the stack to the segv_handler frame and looking at where in
3546	  the code the access happened shows that it happened near line 110 of
3547	  block_dev.c:
3548	
3549	
3550	
3551	
3552	
3553	
3554	
3555	
3556	
3557	  (gdb) up
3558	  #1  0x1007584d in __sleep (seconds=1000000)
3559	      at ../sysdeps/unix/sysv/linux/sleep.c:78
3560	  ../sysdeps/unix/sysv/linux/sleep.c:78: No such file or directory.
3561	  (gdb)
3562	  #2  0x1006ce9a in stop () at user_util.c:191
3563	  191       while(1) sleep(1000000);
3564	  (gdb)
3565	  #3  0x1006bf88 in segv (address=1342179328, is_write=2) at trap_kern.c:31
3566	  31          KERN_UNTESTED();
3567	  (gdb)
3568	  #4  0x1006c628 in segv_handler (sc=0x5006eaf8) at trap_user.c:174
3569	  174       segv(sc->cr2, sc->err & 2);
3570	  (gdb) p *sc
3571	  $1 = {gs = 0, __gsh = 0, fs = 0, __fsh = 0, es = 43, __esh = 0, ds = 43,
3572	    __dsh = 0, edi = 1342179328, esi = 134973440, ebp = 1342631484,
3573	    esp = 1342630864, ebx = 256, edx = 0, ecx = 256, eax = 1024, trapno = 14,
3574	    err = 6, eip = 268550834, cs = 35, __csh = 0, eflags = 66070,
3575	    esp_at_signal = 1342630864, ss = 43, __ssh = 0, fpstate = 0x0, oldmask = 0,
3576	    cr2 = 1342179328}
3577	  (gdb) p (void *)268550834
3578	  $2 = (void *) 0x1001c2b2
3579	  (gdb) i sym $2
3580	  block_write + 1090 in section .text
3581	  (gdb) i line *$2
3582	  Line 209 of "/home/dike/linux/2.3.26/um/include/asm/arch/string.h"
3583	     starts at address 0x1001c2a1 <block_write+1073>
3584	     and ends at 0x1001c2bf <block_write+1103>.
3585	  (gdb) i line *0x1001c2c0
3586	  Line 110 of "block_dev.c" starts at address 0x1001c2bf <block_write+1103>
3587	     and ends at 0x1001c2e3 <block_write+1139>.
3588	
3589	
3590	
3591	
3592	
3593	  Looking at the source shows that the fault happened during a call to
3594	  copy_from_user to copy the data into the kernel:
3595	
3596	
3597	       107             count -= chars;
3598	       108             copy_from_user(p,buf,chars);
3599	       109             p += chars;
3600	       110             buf += chars;
3601	
3602	
3603	
3604	
3605	
3606	  p is the pointer which must contain 0x50000800, since buf contains
3607	  0x80b8800 (frame 8 above).  It is defined as:
3608	
3609	
3610	                       p = offset + bh->b_data;
3611	
3612	
3613	
3614	
3615	
3616	  I need to figure out what bh is, and it just so happens that bh is
3617	  passed as an argument to mark_buffer_uptodate and mark_buffer_dirty a
3618	  few lines later, so I do a little disassembly:
3619	
3620	
3621	
3622	
3623	  (gdb) disas 0x1001c2bf 0x1001c2e0
3624	  Dump of assembler code from 0x1001c2bf to 0x1001c2d0:
3625	  0x1001c2bf <block_write+1103>:  addl   %eax,0xc(%ebp)
3626	  0x1001c2c2 <block_write+1106>:  movl   0xfffffdd4(%ebp),%edx
3627	  0x1001c2c8 <block_write+1112>:  btsl   $0x0,0x18(%edx)
3628	  0x1001c2cd <block_write+1117>:  btsl   $0x1,0x18(%edx)
3629	  0x1001c2d2 <block_write+1122>:  sbbl   %ecx,%ecx
3630	  0x1001c2d4 <block_write+1124>:  testl  %ecx,%ecx
3631	  0x1001c2d6 <block_write+1126>:  jne    0x1001c2e3 <block_write+1139>
3632	  0x1001c2d8 <block_write+1128>:  pushl  $0x0
3633	  0x1001c2da <block_write+1130>:  pushl  %edx
3634	  0x1001c2db <block_write+1131>:  call   0x1001819c <__mark_buffer_dirty>
3635	  End of assembler dump.
3636	
3637	
3638	
3639	
3640	
3641	  At that point, bh is in %edx (address 0x1001c2da), which is calculated
3642	  at 0x1001c2c2 as %ebp + 0xfffffdd4, so I figure exactly what that is,
3643	  taking %ebp from the sigcontext_struct above:
3644	
3645	
3646	       (gdb) p (void *)1342631484
3647	       $5 = (void *) 0x5006ee3c
3648	       (gdb) p 0x5006ee3c+0xfffffdd4
3649	       $6 = 1342630928
3650	       (gdb) p (void *)$6
3651	       $7 = (void *) 0x5006ec10
3652	       (gdb) p *((void **)$7)
3653	       $8 = (void *) 0x50100200
3654	
3655	
3656	
3657	
3658	
3659	  Now, I look at the structure to see what's in it, and particularly,
3660	  what its b_data field contains:
3661	
3662	
3663	       (gdb) p *((struct buffer_head *)0x50100200)
3664	       $13 = {b_next = 0x50289380, b_blocknr = 49405, b_size = 1024, b_list = 0,
3665	         b_dev = 15872, b_count = {counter = 1}, b_rdev = 15872, b_state = 24,
3666	         b_flushtime = 0, b_next_free = 0x501001a0, b_prev_free = 0x50100260,
3667	         b_this_page = 0x501001a0, b_reqnext = 0x0, b_pprev = 0x507fcf58,
3668	         b_data = 0x50000800 "", b_page = 0x50004000,
3669	         b_end_io = 0x10017f60 <end_buffer_io_sync>, b_dev_id = 0x0,
3670	         b_rsector = 98810, b_wait = {lock = <optimized out or zero length>,
3671	           task_list = {next = 0x50100248, prev = 0x50100248}, __magic = 1343226448,
3672	           __creator = 0}, b_kiobuf = 0x0}
3673	
3674	
3675	
3676	
3677	
3678	  The b_data field is indeed 0x50000800, so the question becomes how
3679	  that happened.  The rest of the structure looks fine, so this probably
3680	  is not a case of data corruption.  It happened on purpose somehow.
3681	
3682	
3683	  The b_page field is a pointer to the page_struct representing the
3684	  0x50000000 page.  Looking at it shows the kernel's idea of the state
3685	  of that page:
3686	
3687	
3688	
3689	  (gdb) p *$13.b_page
3690	  $17 = {list = {next = 0x50004a5c, prev = 0x100c5174}, mapping = 0x0,
3691	    index = 0, next_hash = 0x0, count = {counter = 1}, flags = 132, lru = {
3692	      next = 0x50008460, prev = 0x50019350}, wait = {
3693	      lock = <optimized out or zero length>, task_list = {next = 0x50004024,
3694	        prev = 0x50004024}, __magic = 1342193708, __creator = 0},
3695	    pprev_hash = 0x0, buffers = 0x501002c0, virtual = 1342177280,
3696	    zone = 0x100c5160}
3697	
3698	
3699	
3700	
3701	
3702	  Some sanity-checking: the virtual field shows the "virtual" address of
3703	  this page, which in this kernel is the same as its "physical" address,
3704	  and the page_struct itself should be mem_map[0], since it represents
3705	  the first page of memory:
3706	
3707	
3708	
3709	       (gdb) p (void *)1342177280
3710	       $18 = (void *) 0x50000000
3711	       (gdb) p mem_map
3712	       $19 = (mem_map_t *) 0x50004000
3713	
3714	
3715	
3716	
3717	
3718	  These check out fine.
3719	
3720	
3721	  Now to check out the page_struct itself.  In particular, the flags
3722	  field shows whether the page is considered free or not:
3723	
3724	
3725	       (gdb) p (void *)132
3726	       $21 = (void *) 0x84
3727	
3728	
3729	
3730	
3731	
3732	  The "reserved" bit is the high bit, which is definitely not set, so
3733	  the kernel considers the signal stack page to be free and available to
3734	  be used.
3735	
3736	
3737	  At this point, I jump to conclusions and start looking at my early
3738	  boot code, because that's where that page is supposed to be reserved.
3739	
3740	
3741	  In my setup_arch procedure, I have the following code which looks just
3742	  fine:
3743	
3744	
3745	
3746	       bootmap_size = init_bootmem(start_pfn, end_pfn - start_pfn);
3747	       free_bootmem(__pa(low_physmem) + bootmap_size, high_physmem - low_physmem);
3748	
3749	
3750	
3751	
3752	
3753	  Two stack pages have already been allocated, and low_physmem points to
3754	  the third page, which is the beginning of free memory.
3755	  The init_bootmem call declares the entire memory to the boot memory
3756	  manager, which marks it all reserved.  The free_bootmem call frees up
3757	  all of it, except for the first two pages.  This looks correct to me.
3758	
3759	
3760	  So, I decide to see init_bootmem run and make sure that it is marking
3761	  those first two pages as reserved.  I never get that far.
3762	
3763	
3764	  Stepping into init_bootmem, and looking at bootmem_map before looking
3765	  at what it contains shows the following:
3766	
3767	
3768	
3769	       (gdb) p bootmem_map
3770	       $3 = (void *) 0x50000000
3771	
3772	
3773	
3774	
3775	
3776	  Aha!  The light dawns.  That first page is doing double duty as a
3777	  stack and as the boot memory map.  The last thing that the boot memory
3778	  manager does is to free the pages used by its memory map, so this page
3779	  is getting freed even its marked as reserved.
3780	
3781	
3782	  The fix was to initialize the boot memory manager before allocating
3783	  those two stack pages, and then allocate them through the boot memory
3784	  manager.  After doing this, and fixing a couple of subsequent buglets,
3785	  the stack corruption problem disappeared.
3786	
3787	
3788	
3789	
3790	
3791	  13.  What to do when UML doesn't work
3792	
3793	
3794	
3795	
3796	  13.1.  Strange compilation errors when you build from source
3797	
3798	  As of test11, it is necessary to have "ARCH=um" in the environment or
3799	  on the make command line for all steps in building UML, including
3800	  clean, distclean, or mrproper, config, menuconfig, or xconfig, dep,
3801	  and linux.  If you forget for any of them, the i386 build seems to
3802	  contaminate the UML build.  If this happens, start from scratch with
3803	
3804	
3805	       host%
3806	       make mrproper ARCH=um
3807	
3808	
3809	
3810	
3811	  and repeat the build process with ARCH=um on all the steps.
3812	
3813	
3814	  See ``Compiling the kernel and modules''  for more details.
3815	
3816	
3817	  Another cause of strange compilation errors is building UML in
3818	  /usr/src/linux.  If you do this, the first thing you need to do is
3819	  clean up the mess you made.  The /usr/src/linux/asm link will now
3820	  point to /usr/src/linux/asm-um.  Make it point back to
3821	  /usr/src/linux/asm-i386.  Then, move your UML pool someplace else and
3822	  build it there.  Also see below, where a more specific set of symptoms
3823	  is described.
3824	
3825	
3826	
3827	  13.3.  A variety of panics and hangs with /tmp on a reiserfs  filesys-
3828	  tem
3829	
3830	  I saw this on reiserfs 3.5.21 and it seems to be fixed in 3.5.27.
3831	  Panics preceded by
3832	
3833	
3834	       Detaching pid nnnn
3835	
3836	
3837	
3838	  are diagnostic of this problem.  This is a reiserfs bug which causes a
3839	  thread to occasionally read stale data from a mmapped page shared with
3840	  another thread.  The fix is to upgrade the filesystem or to have /tmp
3841	  be an ext2 filesystem.
3842	
3843	
3844	
3845	  13.4.  The compile fails with errors about conflicting types for
3846	  'open', 'dup', and 'waitpid'
3847	
3848	  This happens when you build in /usr/src/linux.  The UML build makes
3849	  the include/asm link point to include/asm-um.  /usr/include/asm points
3850	  to /usr/src/linux/include/asm, so when that link gets moved, files
3851	  which need to include the asm-i386 versions of headers get the
3852	  incompatible asm-um versions.  The fix is to move the include/asm link
3853	  back to include/asm-i386 and to do UML builds someplace else.
3854	
3855	
3856	
3857	  13.5.  UML doesn't work when /tmp is an NFS filesystem
3858	
3859	  This seems to be a similar situation with the ReiserFS problem above.
3860	  Some versions of NFS seems not to handle mmap correctly, which UML
3861	  depends on.  The workaround is have /tmp be a non-NFS directory.
3862	
3863	
3864	  13.6.  UML hangs on boot when compiled with gprof support
3865	
3866	  If you build UML with gprof support and, early in the boot, it does
3867	  this
3868	
3869	
3870	       kernel BUG at page_alloc.c:100!
3871	
3872	
3873	
3874	
3875	  you have a buggy gcc.  You can work around the problem by removing
3876	  UM_FASTCALL from CFLAGS in arch/um/Makefile-i386.  This will open up
3877	  another bug, but that one is fairly hard to reproduce.
3878	
3879	
3880	
3881	  13.7.  syslogd dies with a SIGTERM on startup
3882	
3883	  The exact boot error depends on the distribution that you're booting,
3884	  but Debian produces this:
3885	
3886	
3887	       /etc/rc2.d/S10sysklogd: line 49:    93 Terminated
3888	       start-stop-daemon --start --quiet --exec /sbin/syslogd -- $SYSLOGD
3889	
3890	
3891	
3892	
3893	  This is a syslogd bug.  There's a race between a parent process
3894	  installing a signal handler and its child sending the signal.  See
3895	  this uml-devel post <http://www.geocrawler.com/lists/3/Source-
3896	  Forge/709/0/6612801>  for the details.
3897	
3898	
3899	
3900	  13.8.  TUN/TAP networking doesn't work on a 2.4 host
3901	
3902	  There are a couple of problems which were
3903	  <http://www.geocrawler.com/lists/3/SourceForge/597/0/> name="pointed
3904	  out">  by Tim Robinson <timro at trkr dot net>
3905	
3906	  o  It doesn't work on hosts running 2.4.7 (or thereabouts) or earlier.
3907	     The fix is to upgrade to something more recent and then read the
3908	     next item.
3909	
3910	  o  If you see
3911	
3912	
3913	       File descriptor in bad state
3914	
3915	
3916	
3917	  when you bring up the device inside UML, you have a header mismatch
3918	  between the original kernel and the upgraded one.  Make /usr/src/linux
3919	  point at the new headers.  This will only be a problem if you build
3920	  uml_net yourself.
3921	
3922	
3923	
3924	  13.9.  You can network to the host but not to other machines on the
3925	  net
3926	
3927	  If you can connect to the host, and the host can connect to UML, but
3928	  you cannot connect to any other machines, then you may need to enable
3929	  IP Masquerading on the host.  Usually this is only experienced when
3930	  using private IP addresses (192.168.x.x or 10.x.x.x) for host/UML
3931	  networking, rather than the public address space that your host is
3932	  connected to.  UML does not enable IP Masquerading, so you will need
3933	  to create a static rule to enable it:
3934	
3935	
3936	       host%
3937	       iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
3938	
3939	
3940	
3941	
3942	  Replace eth0 with the interface that you use to talk to the rest of
3943	  the world.
3944	
3945	
3946	  Documentation on IP Masquerading, and SNAT, can be found at
3947	  www.netfilter.org  <http://www.netfilter.org> .
3948	
3949	
3950	  If you can reach the local net, but not the outside Internet, then
3951	  that is usually a routing problem.  The UML needs a default route:
3952	
3953	
3954	       UML#
3955	       route add default gw gateway IP
3956	
3957	
3958	
3959	
3960	  The gateway IP can be any machine on the local net that knows how to
3961	  reach the outside world.  Usually, this is the host or the local net-
3962	  work's gateway.
3963	
3964	
3965	  Occasionally, we hear from someone who can reach some machines, but
3966	  not others on the same net, or who can reach some ports on other
3967	  machines, but not others.  These are usually caused by strange
3968	  firewalling somewhere between the UML and the other box.  You track
3969	  this down by running tcpdump on every interface the packets travel
3970	  over and see where they disappear.  When you find a machine that takes
3971	  the packets in, but does not send them onward, that's the culprit.
3972	
3973	
3974	
3975	  13.10.  I have no root and I want to scream
3976	
3977	  Thanks to Birgit Wahlich for telling me about this strange one.  It
3978	  turns out that there's a limit of six environment variables on the
3979	  kernel command line.  When that limit is reached or exceeded, argument
3980	  processing stops, which means that the 'root=' argument that UML
3981	  usually adds is not seen.  So, the filesystem has no idea what the
3982	  root device is, so it panics.
3983	
3984	
3985	  The fix is to put less stuff on the command line.  Glomming all your
3986	  setup variables into one is probably the best way to go.
3987	
3988	
3989	
3990	  13.11.  UML build conflict between ptrace.h and ucontext.h
3991	
3992	  On some older systems, /usr/include/asm/ptrace.h and
3993	  /usr/include/sys/ucontext.h define the same names.  So, when they're
3994	  included together, the defines from one completely mess up the parsing
3995	  of the other, producing errors like:
3996	       /usr/include/sys/ucontext.h:47: parse error before
3997	       `10'
3998	
3999	
4000	
4001	
4002	  plus a pile of warnings.
4003	
4004	
4005	  This is a libc botch, which has since been fixed, and I don't see any
4006	  way around it besides upgrading.
4007	
4008	
4009	
4010	  13.12.  The UML BogoMips is exactly half the host's BogoMips
4011	
4012	  On i386 kernels, there are two ways of running the loop that is used
4013	  to calculate the BogoMips rating, using the TSC if it's there or using
4014	  a one-instruction loop.  The TSC produces twice the BogoMips as the
4015	  loop.  UML uses the loop, since it has nothing resembling a TSC, and
4016	  will get almost exactly the same BogoMips as a host using the loop.
4017	  However, on a host with a TSC, its BogoMips will be double the loop
4018	  BogoMips, and therefore double the UML BogoMips.
4019	
4020	
4021	
4022	  13.13.  When you run UML, it immediately segfaults
4023	
4024	  If the host is configured with the 2G/2G address space split, that's
4025	  why.  See ``UML on 2G/2G hosts''  for the details on getting UML to
4026	  run on your host.
4027	
4028	
4029	
4030	  13.14.  xterms appear, then immediately disappear
4031	
4032	  If you're running an up to date kernel with an old release of
4033	  uml_utilities, the port-helper program will not work properly, so
4034	  xterms will exit straight after they appear. The solution is to
4035	  upgrade to the latest release of uml_utilities.  Usually this problem
4036	  occurs when you have installed a packaged release of UML then compiled
4037	  your own development kernel without upgrading the uml_utilities from
4038	  the source distribution.
4039	
4040	
4041	
4042	  13.15.  Any other panic, hang, or strange behavior
4043	
4044	  If you're seeing truly strange behavior, such as hangs or panics that
4045	  happen in random places, or you try running the debugger to see what's
4046	  happening and it acts strangely, then it could be a problem in the
4047	  host kernel.  If you're not running a stock Linus or -ac kernel, then
4048	  try that.  An early version of the preemption patch and a 2.4.10 SuSE
4049	  kernel have caused very strange problems in UML.
4050	
4051	
4052	  Otherwise, let me know about it.  Send a message to one of the UML
4053	  mailing lists - either the developer list - user-mode-linux-devel at
4054	  lists dot sourceforge dot net (subscription info) or the user list -
4055	  user-mode-linux-user at lists dot sourceforge do net (subscription
4056	  info), whichever you prefer.  Don't assume that everyone knows about
4057	  it and that a fix is imminent.
4058	
4059	
4060	  If you want to be super-helpful, read ``Diagnosing Problems'' and
4061	  follow the instructions contained therein.
4062	  14.  Diagnosing Problems
4063	
4064	
4065	  If you get UML to crash, hang, or otherwise misbehave, you should
4066	  report this on one of the project mailing lists, either the developer
4067	  list - user-mode-linux-devel at lists dot sourceforge dot net
4068	  (subscription info) or the user list - user-mode-linux-user at lists
4069	  dot sourceforge dot net (subscription info).  When you do, it is
4070	  likely that I will want more information.  So, it would be helpful to
4071	  read the stuff below, do whatever is applicable in your case, and
4072	  report the results to the list.
4073	
4074	
4075	  For any diagnosis, you're going to need to build a debugging kernel.
4076	  The binaries from this site aren't debuggable.  If you haven't done
4077	  this before, read about ``Compiling the kernel and modules''  and
4078	  ``Kernel debugging''  UML first.
4079	
4080	
4081	  14.1.  Case 1 : Normal kernel panics
4082	
4083	  The most common case is for a normal thread to panic.  To debug this,
4084	  you will need to run it under the debugger (add 'debug' to the command
4085	  line).  An xterm will start up with gdb running inside it.  Continue
4086	  it when it stops in start_kernel and make it crash.  Now ^C gdb and
4087	
4088	
4089	  If the panic was a "Kernel mode fault", then there will be a segv
4090	  frame on the stack and I'm going to want some more information.  The
4091	  stack might look something like this:
4092	
4093	
4094	       (UML gdb)  backtrace
4095	       #0  0x1009bf76 in __sigprocmask (how=1, set=0x5f347940, oset=0x0)
4096	           at ../sysdeps/unix/sysv/linux/sigprocmask.c:49
4097	       #1  0x10091411 in change_sig (signal=10, on=1) at process.c:218
4098	       #2  0x10094785 in timer_handler (sig=26) at time_kern.c:32
4099	       #3  0x1009bf38 in __restore ()
4100	           at ../sysdeps/unix/sysv/linux/i386/sigaction.c:125
4101	       #4  0x1009534c in segv (address=8, ip=268849158, is_write=2, is_user=0)
4102	           at trap_kern.c:66
4103	       #5  0x10095c04 in segv_handler (sig=11) at trap_user.c:285
4104	       #6  0x1009bf38 in __restore ()
4105	
4106	
4107	
4108	
4109	  I'm going to want to see the symbol and line information for the value
4110	  of ip in the segv frame.  In this case, you would do the following:
4111	
4112	
4113	       (UML gdb)  i sym 268849158
4114	
4115	
4116	
4117	
4118	  and
4119	
4120	
4121	       (UML gdb)  i line *268849158
4122	
4123	
4124	
4125	
4126	  The reason for this is the __restore frame right above the segv_han-
4127	  dler frame is hiding the frame that actually segfaulted.  So, I have
4128	  to get that information from the faulting ip.
4129	
4130	
4131	  14.2.  Case 2 : Tracing thread panics
4132	
4133	  The less common and more painful case is when the tracing thread
4134	  panics.  In this case, the kernel debugger will be useless because it
4135	  needs a healthy tracing thread in order to work.  The first thing to
4136	  do is get a backtrace from the tracing thread.  This is done by
4137	  figuring out what its pid is, firing up gdb, and attaching it to that
4138	  pid.  You can figure out the tracing thread pid by looking at the
4139	  first line of the console output, which will look like this:
4140	
4141	
4142	       tracing thread pid = 15851
4143	
4144	
4145	
4146	
4147	  or by running ps on the host and finding the line that looks like
4148	  this:
4149	
4150	
4151	       jdike 15851 4.5 0.4 132568 1104 pts/0 S 21:34 0:05 ./linux [(tracing thread)]
4152	
4153	
4154	
4155	
4156	  If the panic was 'segfault in signals', then follow the instructions
4157	  above for collecting information about the location of the seg fault.
4158	
4159	
4160	  If the tracing thread flaked out all by itself, then send that
4161	  backtrace in and wait for our crack debugging team to fix the problem.
4162	
4163	
4164	  14.3.  Case 3 : Tracing thread panics caused by other threads
4165	
4166	  However, there are cases where the misbehavior of another thread
4167	  caused the problem.  The most common panic of this type is:
4168	
4169	
4170	       wait_for_stop failed to wait for  <pid>  to stop with  <signal number>
4171	
4172	
4173	
4174	
4175	  In this case, you'll need to get a backtrace from the process men-
4176	  tioned in the panic, which is complicated by the fact that the kernel
4177	  debugger is defunct and without some fancy footwork, another gdb can't
4178	  attach to it.  So, this is how the fancy footwork goes:
4179	
4180	  In a shell:
4181	
4182	
4183	       host% kill -STOP pid
4184	
4185	
4186	
4187	
4188	  Run gdb on the tracing thread as described in case 2 and do:
4189	
4190	
4191	       (host gdb)  call detach(pid)
4192	
4193	
4194	  If you get a segfault, do it again.  It always works the second time.
4195	
4196	  Detach from the tracing thread and attach to that other thread:
4197	
4198	
4199	       (host gdb)  detach
4200	
4201	
4202	
4203	
4204	
4205	
4206	       (host gdb)  attach pid
4207	
4208	
4209	
4210	
4211	  If gdb hangs when attaching to that process, go back to a shell and
4212	  do:
4213	
4214	
4215	       host%
4216	       kill -CONT pid
4217	
4218	
4219	
4220	
4221	  And then get the backtrace:
4222	
4223	
4224	       (host gdb)  backtrace
4225	
4226	
4227	
4228	
4229	
4230	  14.4.  Case 4 : Hangs
4231	
4232	  Hangs seem to be fairly rare, but they sometimes happen.  When a hang
4233	  happens, we need a backtrace from the offending process.  Run the
4234	  kernel debugger as described in case 1 and get a backtrace.  If the
4235	  current process is not the idle thread, then send in the backtrace.
4236	  You can tell that it's the idle thread if the stack looks like this:
4237	
4238	
4239	       #0  0x100b1401 in __libc_nanosleep ()
4240	       #1  0x100a2885 in idle_sleep (secs=10) at time.c:122
4241	       #2  0x100a546f in do_idle () at process_kern.c:445
4242	       #3  0x100a5508 in cpu_idle () at process_kern.c:471
4243	       #4  0x100ec18f in start_kernel () at init/main.c:592
4244	       #5  0x100a3e10 in start_kernel_proc (unused=0x0) at um_arch.c:71
4245	       #6  0x100a383f in signal_tramp (arg=0x100a3dd8) at trap_user.c:50
4246	
4247	
4248	
4249	
4250	  If this is the case, then some other process is at fault, and went to
4251	  sleep when it shouldn't have.  Run ps on the host and figure out which
4252	  process should not have gone to sleep and stayed asleep.  Then attach
4253	  to it with gdb and get a backtrace as described in case 3.
4254	
4255	
4256	
4257	
4258	
4259	
4260	  15.  Thanks
4261	
4262	
4263	  A number of people have helped this project in various ways, and this
4264	  page gives recognition where recognition is due.
4265	
4266	
4267	  If you're listed here and you would prefer a real link on your name,
4268	  or no link at all, instead of the despammed email address pseudo-link,
4269	  let me know.
4270	
4271	
4272	  If you're not listed here and you think maybe you should be, please
4273	  let me know that as well.  I try to get everyone, but sometimes my
4274	  bookkeeping lapses and I forget about contributions.
4275	
4276	
4277	  15.1.  Code and Documentation
4278	
4279	  Rusty Russell <rusty at linuxcare.com.au>  -
4280	
4281	  o  wrote the  HOWTO <http://user-mode-
4282	     linux.sourceforge.net/UserModeLinux-HOWTO.html>
4283	
4284	  o  prodded me into making this project official and putting it on
4285	     SourceForge
4286	
4287	  o  came up with the way cool UML logo <http://user-mode-
4288	     linux.sourceforge.net/uml-small.png>
4289	
4290	  o  redid the config process
4291	
4292	
4293	  Peter Moulder <reiter at netspace.net.au>  - Fixed my config and build
4294	  processes, and added some useful code to the block driver
4295	
4296	
4297	  Bill Stearns <wstearns at pobox.com>  -
4298	
4299	  o  HOWTO updates
4300	
4301	  o  lots of bug reports
4302	
4303	  o  lots of testing
4304	
4305	  o  dedicated a box (uml.ists.dartmouth.edu) to support UML development
4306	
4307	  o  wrote the mkrootfs script, which allows bootable filesystems of
4308	     RPM-based distributions to be cranked out
4309	
4310	  o  cranked out a large number of filesystems with said script
4311	
4312	
4313	  Jim Leu <jleu at mindspring.com>  - Wrote the virtual ethernet driver
4314	  and associated usermode tools
4315	
4316	  Lars Brinkhoff <http://lars.nocrew.org/>  - Contributed the ptrace
4317	  proxy from his own  project <http://a386.nocrew.org/> to allow easier
4318	  kernel debugging
4319	
4320	
4321	  Andrea Arcangeli <andrea at suse.de>  - Redid some of the early boot
4322	  code so that it would work on machines with Large File Support
4323	
4324	
4325	  Chris Emerson <http://www.chiark.greenend.org.uk/~cemerson/>  - Did
4326	  the first UML port to Linux/ppc
4327	
4328	
4329	  Harald Welte <laforge at gnumonks.org>  - Wrote the multicast
4330	  transport for the network driver
4331	
4332	
4333	  Jorgen Cederlof - Added special file support to hostfs
4334	
4335	
4336	  Greg Lonnon  <glonnon at ridgerun dot com>  - Changed the ubd driver
4337	  to allow it to layer a COW file on a shared read-only filesystem and
4338	  wrote the iomem emulation support
4339	
4340	
4341	  Henrik Nordstrom <http://hem.passagen.se/hno/>  - Provided a variety
4342	  of patches, fixes, and clues
4343	
4344	
4345	  Lennert Buytenhek - Contributed various patches, a rewrite of the
4346	  network driver, the first implementation of the mconsole driver, and
4347	  did the bulk of the work needed to get SMP working again.
4348	
4349	
4350	  Yon Uriarte - Fixed the TUN/TAP network backend while I slept.
4351	
4352	
4353	  Adam Heath - Made a bunch of nice cleanups to the initialization code,
4354	  plus various other small patches.
4355	
4356	
4357	  Matt Zimmerman - Matt volunteered to be the UML Debian maintainer and
4358	  is doing a real nice job of it.  He also noticed and fixed a number of
4359	  actually and potentially exploitable security holes in uml_net.  Plus
4360	  the occasional patch.  I like patches.
4361	
4362	
4363	  James McMechan - James seems to have taken over maintenance of the ubd
4364	  driver and is doing a nice job of it.
4365	
4366	
4367	  Chandan Kudige - wrote the umlgdb script which automates the reloading
4368	  of module symbols.
4369	
4370	
4371	  Steve Schmidtke - wrote the UML slirp transport and hostaudio drivers,
4372	  enabling UML processes to access audio devices on the host. He also
4373	  submitted patches for the slip transport and lots of other things.
4374	
4375	
4376	  David Coulson <http://davidcoulson.net>  -
4377	
4378	  o  Set up the usermodelinux.org <http://usermodelinux.org>  site,
4379	     which is a great way of keeping the UML user community on top of
4380	     UML goings-on.
4381	
4382	  o  Site documentation and updates
4383	
4384	  o  Nifty little UML management daemon  UMLd
4385	     <http://uml.openconsultancy.com/umld/>
4386	
4387	  o  Lots of testing and bug reports
4388	
4389	
4390	
4391	
4392	  15.2.  Flushing out bugs
4393	
4394	
4395	
4396	  o  Yuri Pudgorodsky
4397	
4398	  o  Gerald Britton
4399	
4400	  o  Ian Wehrman
4401	
4402	  o  Gord Lamb
4403	
4404	  o  Eugene Koontz
4405	
4406	  o  John H. Hartman
4407	
4408	  o  Anders Karlsson
4409	
4410	  o  Daniel Phillips
4411	
4412	  o  John Fremlin
4413	
4414	  o  Rainer Burgstaller
4415	
4416	  o  James Stevenson
4417	
4418	  o  Matt Clay
4419	
4420	  o  Cliff Jefferies
4421	
4422	  o  Geoff Hoff
4423	
4424	  o  Lennert Buytenhek
4425	
4426	  o  Al Viro
4427	
4428	  o  Frank Klingenhoefer
4429	
4430	  o  Livio Baldini Soares
4431	
4432	  o  Jon Burgess
4433	
4434	  o  Petru Paler
4435	
4436	  o  Paul
4437	
4438	  o  Chris Reahard
4439	
4440	  o  Sverker Nilsson
4441	
4442	  o  Gong Su
4443	
4444	  o  johan verrept
4445	
4446	  o  Bjorn Eriksson
4447	
4448	  o  Lorenzo Allegrucci
4449	
4450	  o  Muli Ben-Yehuda
4451	
4452	  o  David Mansfield
4453	
4454	  o  Howard Goff
4455	
4456	  o  Mike Anderson
4457	
4458	  o  John Byrne
4459	
4460	  o  Sapan J. Batia
4461	
4462	  o  Iris Huang
4463	
4464	  o  Jan Hudec
4465	
4466	  o  Voluspa
4467	
4468	
4469	
4470	
4471	  15.3.  Buglets and clean-ups
4472	
4473	
4474	
4475	  o  Dave Zarzycki
4476	
4477	  o  Adam Lazur
4478	
4479	  o  Boria Feigin
4480	
4481	  o  Brian J. Murrell
4482	
4483	  o  JS
4484	
4485	  o  Roman Zippel
4486	
4487	  o  Wil Cooley
4488	
4489	  o  Ayelet Shemesh
4490	
4491	  o  Will Dyson
4492	
4493	  o  Sverker Nilsson
4494	
4495	  o  dvorak
4496	
4497	  o  v.naga srinivas
4498	
4499	  o  Shlomi Fish
4500	
4501	  o  Roger Binns
4502	
4503	  o  johan verrept
4504	
4505	  o  MrChuoi
4506	
4507	  o  Peter Cleve
4508	
4509	  o  Vincent Guffens
4510	
4511	  o  Nathan Scott
4512	
4513	  o  Patrick Caulfield
4514	
4515	  o  jbearce
4516	
4517	  o  Catalin Marinas
4518	
4519	  o  Shane Spencer
4520	
4521	  o  Zou Min
4522	
4523	
4524	  o  Ryan Boder
4525	
4526	  o  Lorenzo Colitti
4527	
4528	  o  Gwendal Grignou
4529	
4530	  o  Andre' Breiler
4531	
4532	  o  Tsutomu Yasuda
4533	
4534	
4535	
4536	  15.4.  Case Studies
4537	
4538	
4539	  o  Jon Wright
4540	
4541	  o  William McEwan
4542	
4543	  o  Michael Richardson
4544	
4545	
4546	
4547	  15.5.  Other contributions
4548	
4549	
4550	  Bill Carr <Bill.Carr at compaq.com>  made the Red Hat mkrootfs script
4551	  work with RH 6.2.
4552	
4553	  Michael Jennings <mikejen at hevanet.com>  sent in some material which
4554	  is now gracing the top of the  index  page <http://user-mode-
4555	  linux.sourceforge.net/>  of this site.
4556	
4557	  SGI <http://www.sgi.com>  (and more specifically Ralf Baechle <ralf at
4558	  uni-koblenz.de> ) gave me an account on oss.sgi.com
4559	  <http://www.oss.sgi.com> .  The bandwidth there made it possible to
4560	  produce most of the filesystems available on the project download
4561	  page.
4562	
4563	  Laurent Bonnaud <Laurent.Bonnaud at inpg.fr>  took the old grotty
4564	  Debian filesystem that I've been distributing and updated it to 2.2.
4565	  It is now available by itself here.
4566	
4567	  Rik van Riel gave me some ftp space on ftp.nl.linux.org so I can make
4568	  releases even when Sourceforge is broken.
4569	
4570	  Rodrigo de Castro looked at my broken pte code and told me what was
4571	  wrong with it, letting me fix a long-standing (several weeks) and
4572	  serious set of bugs.
4573	
4574	  Chris Reahard built a specialized root filesystem for running a DNS
4575	  server jailed inside UML.  It's available from the download
4576	  <http://user-mode-linux.sourceforge.net/dl-sf.html>  page in the Jail
4577	  Filesystems section.
4578	
4579	
4580	
4581	
4582	
4583	
4584	
4585	
4586	
4587	
4588	
Hide Line Numbers


About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog