About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Documentation / x86 / x86_64 / fake-numa-for-cpusets


Based on kernel version 4.16.1. Page generated on 2018-04-09 11:53 EST.

1	Using numa=fake and CPUSets for Resource Management
2	Written by David Rientjes <rientjes@cs.washington.edu>
3	
4	This document describes how the numa=fake x86_64 command-line option can be used
5	in conjunction with cpusets for coarse memory management.  Using this feature,
6	you can create fake NUMA nodes that represent contiguous chunks of memory and
7	assign them to cpusets and their attached tasks.  This is a way of limiting the
8	amount of system memory that are available to a certain class of tasks.
9	
10	For more information on the features of cpusets, see
11	Documentation/cgroup-v1/cpusets.txt.
12	There are a number of different configurations you can use for your needs.  For
13	more information on the numa=fake command line option and its various ways of
14	configuring fake nodes, see Documentation/x86/x86_64/boot-options.txt.
15	
16	For the purposes of this introduction, we'll assume a very primitive NUMA
17	emulation setup of "numa=fake=4*512,".  This will split our system memory into
18	four equal chunks of 512M each that we can now use to assign to cpusets.  As
19	you become more familiar with using this combination for resource control,
20	you'll determine a better setup to minimize the number of nodes you have to deal
21	with.
22	
23	A machine may be split as follows with "numa=fake=4*512," as reported by dmesg:
24	
25		Faking node 0 at 0000000000000000-0000000020000000 (512MB)
26		Faking node 1 at 0000000020000000-0000000040000000 (512MB)
27		Faking node 2 at 0000000040000000-0000000060000000 (512MB)
28		Faking node 3 at 0000000060000000-0000000080000000 (512MB)
29		...
30		On node 0 totalpages: 130975
31		On node 1 totalpages: 131072
32		On node 2 totalpages: 131072
33		On node 3 totalpages: 131072
34	
35	Now following the instructions for mounting the cpusets filesystem from
36	Documentation/cgroup-v1/cpusets.txt, you can assign fake nodes (i.e. contiguous memory
37	address spaces) to individual cpusets:
38	
39		[root@xroads /]# mkdir exampleset
40		[root@xroads /]# mount -t cpuset none exampleset
41		[root@xroads /]# mkdir exampleset/ddset
42		[root@xroads /]# cd exampleset/ddset
43		[root@xroads /exampleset/ddset]# echo 0-1 > cpus
44		[root@xroads /exampleset/ddset]# echo 0-1 > mems
45	
46	Now this cpuset, 'ddset', will only allowed access to fake nodes 0 and 1 for
47	memory allocations (1G).
48	
49	You can now assign tasks to these cpusets to limit the memory resources
50	available to them according to the fake nodes assigned as mems:
51	
52		[root@xroads /exampleset/ddset]# echo $$ > tasks
53		[root@xroads /exampleset/ddset]# dd if=/dev/zero of=tmp bs=1024 count=1G
54		[1] 13425
55	
56	Notice the difference between the system memory usage as reported by
57	/proc/meminfo between the restricted cpuset case above and the unrestricted
58	case (i.e. running the same 'dd' command without assigning it to a fake NUMA
59	cpuset):
60					Unrestricted	Restricted
61		MemTotal:		3091900 kB	3091900 kB
62		MemFree:		  42113 kB	1513236 kB
63	
64	This allows for coarse memory management for the tasks you assign to particular
65	cpusets.  Since cpusets can form a hierarchy, you can create some pretty
66	interesting combinations of use-cases for various classes of tasks for your
67	memory management needs.
Hide Line Numbers


About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog