About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Documentation / vm / split_page_table_lock




Custom Search

Based on kernel version 3.16. Page generated on 2014-08-06 21:41 EST.

1	Split page table lock
2	=====================
3	
4	Originally, mm->page_table_lock spinlock protected all page tables of the
5	mm_struct. But this approach leads to poor page fault scalability of
6	multi-threaded applications due high contention on the lock. To improve
7	scalability, split page table lock was introduced.
8	
9	With split page table lock we have separate per-table lock to serialize
10	access to the table. At the moment we use split lock for PTE and PMD
11	tables. Access to higher level tables protected by mm->page_table_lock.
12	
13	There are helpers to lock/unlock a table and other accessor functions:
14	 - pte_offset_map_lock()
15		maps pte and takes PTE table lock, returns pointer to the taken
16		lock;
17	 - pte_unmap_unlock()
18		unlocks and unmaps PTE table;
19	 - pte_alloc_map_lock()
20		allocates PTE table if needed and take the lock, returns pointer
21		to taken lock or NULL if allocation failed;
22	 - pte_lockptr()
23		returns pointer to PTE table lock;
24	 - pmd_lock()
25		takes PMD table lock, returns pointer to taken lock;
26	 - pmd_lockptr()
27		returns pointer to PMD table lock;
28	
29	Split page table lock for PTE tables is enabled compile-time if
30	CONFIG_SPLIT_PTLOCK_CPUS (usually 4) is less or equal to NR_CPUS.
31	If split lock is disabled, all tables guaded by mm->page_table_lock.
32	
33	Split page table lock for PMD tables is enabled, if it's enabled for PTE
34	tables and the architecture supports it (see below).
35	
36	Hugetlb and split page table lock
37	---------------------------------
38	
39	Hugetlb can support several page sizes. We use split lock only for PMD
40	level, but not for PUD.
41	
42	Hugetlb-specific helpers:
43	 - huge_pte_lock()
44		takes pmd split lock for PMD_SIZE page, mm->page_table_lock
45		otherwise;
46	 - huge_pte_lockptr()
47		returns pointer to table lock;
48	
49	Support of split page table lock by an architecture
50	---------------------------------------------------
51	
52	There's no need in special enabling of PTE split page table lock:
53	everything required is done by pgtable_page_ctor() and pgtable_page_dtor(),
54	which must be called on PTE table allocation / freeing.
55	
56	Make sure the architecture doesn't use slab allocator for page table
57	allocation: slab uses page->slab_cache and page->first_page for its pages.
58	These fields share storage with page->ptl.
59	
60	PMD split lock only makes sense if you have more than two page table
61	levels.
62	
63	PMD split lock enabling requires pgtable_pmd_page_ctor() call on PMD table
64	allocation and pgtable_pmd_page_dtor() on freeing.
65	
66	Allocation usually happens in pmd_alloc_one(), freeing in pmd_free() and
67	pmd_free_tlb(), but make sure you cover all PMD table allocation / freeing
68	paths: i.e X86_PAE preallocate few PMDs on pgd_alloc().
69	
70	With everything in place you can set CONFIG_ARCH_ENABLE_SPLIT_PMD_PTLOCK.
71	
72	NOTE: pgtable_page_ctor() and pgtable_pmd_page_ctor() can fail -- it must
73	be handled properly.
74	
75	page->ptl
76	---------
77	
78	page->ptl is used to access split page table lock, where 'page' is struct
79	page of page containing the table. It shares storage with page->private
80	(and few other fields in union).
81	
82	To avoid increasing size of struct page and have best performance, we use a
83	trick:
84	 - if spinlock_t fits into long, we use page->ptr as spinlock, so we
85	   can avoid indirect access and save a cache line.
86	 - if size of spinlock_t is bigger then size of long, we use page->ptl as
87	   pointer to spinlock_t and allocate it dynamically. This allows to use
88	   split lock with enabled DEBUG_SPINLOCK or DEBUG_LOCK_ALLOC, but costs
89	   one more cache line for indirect access;
90	
91	The spinlock_t allocated in pgtable_page_ctor() for PTE table and in
92	pgtable_pmd_page_ctor() for PMD table.
93	
94	Please, never access page->ptl directly -- use appropriate helper.
Hide Line Numbers
About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Information is copyright its respective author. All material is available from the Linux Kernel Source distributed under a GPL License. This page is provided as a free service by mjmwired.net.