About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Documentation / development-process / 3.Early-stage

Custom Search

Based on kernel version 4.8. Page generated on 2016-10-06 23:10 EST.

3	When contemplating a Linux kernel development project, it can be tempting
4	to jump right in and start coding.  As with any significant project,
5	though, much of the groundwork for success is best laid before the first
6	line of code is written.  Some time spent in early planning and
7	communication can save far more time later on.
12	Like any engineering project, a successful kernel enhancement starts with a
13	clear description of the problem to be solved.  In some cases, this step is
14	easy: when a driver is needed for a specific piece of hardware, for
15	example.  In others, though, it is tempting to confuse the real problem
16	with the proposed solution, and that can lead to difficulties.
18	Consider an example: some years ago, developers working with Linux audio
19	sought a way to run applications without dropouts or other artifacts caused
20	by excessive latency in the system.  The solution they arrived at was a
21	kernel module intended to hook into the Linux Security Module (LSM)
22	framework; this module could be configured to give specific applications
23	access to the realtime scheduler.  This module was implemented and sent to
24	the linux-kernel mailing list, where it immediately ran into problems.
26	To the audio developers, this security module was sufficient to solve their
27	immediate problem.  To the wider kernel community, though, it was seen as a
28	misuse of the LSM framework (which is not intended to confer privileges
29	onto processes which they would not otherwise have) and a risk to system
30	stability.  Their preferred solutions involved realtime scheduling access
31	via the rlimit mechanism for the short term, and ongoing latency reduction
32	work in the long term.
34	The audio community, however, could not see past the particular solution
35	they had implemented; they were unwilling to accept alternatives.  The
36	resulting disagreement left those developers feeling disillusioned with the
37	entire kernel development process; one of them went back to an audio list
38	and posted this:
40		There are a number of very good Linux kernel developers, but they
41		tend to get outshouted by a large crowd of arrogant fools. Trying
42		to communicate user requirements to these people is a waste of
43		time. They are much too "intelligent" to listen to lesser mortals.
45	(http://lwn.net/Articles/131776/).
47	The reality of the situation was different; the kernel developers were far
48	more concerned about system stability, long-term maintenance, and finding
49	the right solution to the problem than they were with a specific module.
50	The moral of the story is to focus on the problem - not a specific solution
51	- and to discuss it with the development community before investing in the
52	creation of a body of code.
54	So, when contemplating a kernel development project, one should obtain
55	answers to a short set of questions:
57	 - What, exactly, is the problem which needs to be solved?
59	 - Who are the users affected by this problem?  Which use cases should the
60	   solution address?
62	 - How does the kernel fall short in addressing that problem now?
64	Only then does it make sense to start considering possible solutions.
69	When planning a kernel development project, it makes great sense to hold
70	discussions with the community before launching into implementation.  Early
71	communication can save time and trouble in a number of ways:
73	 - It may well be that the problem is addressed by the kernel in ways which
74	   you have not understood.  The Linux kernel is large and has a number of
75	   features and capabilities which are not immediately obvious.  Not all
76	   kernel capabilities are documented as well as one might like, and it is
77	   easy to miss things.  Your author has seen the posting of a complete
78	   driver which duplicated an existing driver that the new author had been
79	   unaware of.  Code which reinvents existing wheels is not only wasteful;
80	   it will also not be accepted into the mainline kernel.
82	 - There may be elements of the proposed solution which will not be
83	   acceptable for mainline merging.  It is better to find out about
84	   problems like this before writing the code.
86	 - It's entirely possible that other developers have thought about the
87	   problem; they may have ideas for a better solution, and may be willing
88	   to help in the creation of that solution.
90	Years of experience with the kernel development community have taught a
91	clear lesson: kernel code which is designed and developed behind closed
92	doors invariably has problems which are only revealed when the code is
93	released into the community.  Sometimes these problems are severe,
94	requiring months or years of effort before the code can be brought up to
95	the kernel community's standards.  Some examples include:
97	 - The Devicescape network stack was designed and implemented for
98	   single-processor systems.  It could not be merged into the mainline
99	   until it was made suitable for multiprocessor systems.  Retrofitting
100	   locking and such into code is a difficult task; as a result, the merging
101	   of this code (now called mac80211) was delayed for over a year.
103	 - The Reiser4 filesystem included a number of capabilities which, in the
104	   core kernel developers' opinion, should have been implemented in the
105	   virtual filesystem layer instead.  It also included features which could
106	   not easily be implemented without exposing the system to user-caused
107	   deadlocks.  The late revelation of these problems - and refusal to
108	   address some of them - has caused Reiser4 to stay out of the mainline
109	   kernel.
111	 - The AppArmor security module made use of internal virtual filesystem
112	   data structures in ways which were considered to be unsafe and
113	   unreliable.  This concern (among others) kept AppArmor out of the
114	   mainline for years.
116	In each of these cases, a great deal of pain and extra work could have been
117	avoided with some early discussion with the kernel developers.
120	3.3: WHO DO YOU TALK TO?
122	When developers decide to take their plans public, the next question will
123	be: where do we start?  The answer is to find the right mailing list(s) and
124	the right maintainer.  For mailing lists, the best approach is to look in
125	the MAINTAINERS file for a relevant place to post.  If there is a suitable
126	subsystem list, posting there is often preferable to posting on
127	linux-kernel; you are more likely to reach developers with expertise in the
128	relevant subsystem and the environment may be more supportive.
130	Finding maintainers can be a bit harder.  Again, the MAINTAINERS file is
131	the place to start.  That file tends to not always be up to date, though,
132	and not all subsystems are represented there.  The person listed in the
133	MAINTAINERS file may, in fact, not be the person who is actually acting in
134	that role currently.  So, when there is doubt about who to contact, a
135	useful trick is to use git (and "git log" in particular) to see who is
136	currently active within the subsystem of interest.  Look at who is writing
137	patches, and who, if anybody, is attaching Signed-off-by lines to those
138	patches.  Those are the people who will be best placed to help with a new
139	development project.
141	The task of finding the right maintainer is sometimes challenging enough
142	that the kernel developers have added a script to ease the process:
144		.../scripts/get_maintainer.pl
146	This script will return the current maintainer(s) for a given file or
147	directory when given the "-f" option.  If passed a patch on the
148	command line, it will list the maintainers who should probably receive
149	copies of the patch.  There are a number of options regulating how hard
150	get_maintainer.pl will search for maintainers; please be careful about
151	using the more aggressive options as you may end up including developers
152	who have no real interest in the code you are modifying.
154	If all else fails, talking to Andrew Morton can be an effective way to
155	track down a maintainer for a specific piece of code.
158	3.4: WHEN TO POST?
160	If possible, posting your plans during the early stages can only be
161	helpful.  Describe the problem being solved and any plans that have been
162	made on how the implementation will be done.  Any information you can
163	provide can help the development community provide useful input on the
164	project.
166	One discouraging thing which can happen at this stage is not a hostile
167	reaction, but, instead, little or no reaction at all.  The sad truth of the
168	matter is (1) kernel developers tend to be busy, (2) there is no shortage
169	of people with grand plans and little code (or even prospect of code) to
170	back them up, and (3) nobody is obligated to review or comment on ideas
171	posted by others.  Beyond that, high-level designs often hide problems
172	which are only reviewed when somebody actually tries to implement those
173	designs; for that reason, kernel developers would rather see the code.
175	If a request-for-comments posting yields little in the way of comments, do
176	not assume that it means there is no interest in the project.
177	Unfortunately, you also cannot assume that there are no problems with your
178	idea.  The best thing to do in this situation is to proceed, keeping the
179	community informed as you go.
184	If your work is being done in a corporate environment - as most Linux
185	kernel work is - you must, obviously, have permission from suitably
186	empowered managers before you can post your company's plans or code to a
187	public mailing list.  The posting of code which has not been cleared for
188	release under a GPL-compatible license can be especially problematic; the
189	sooner that a company's management and legal staff can agree on the posting
190	of a kernel development project, the better off everybody involved will be.
192	Some readers may be thinking at this point that their kernel work is
193	intended to support a product which does not yet have an officially
194	acknowledged existence.  Revealing their employer's plans on a public
195	mailing list may not be a viable option.  In cases like this, it is worth
196	considering whether the secrecy is really necessary; there is often no real
197	need to keep development plans behind closed doors.
199	That said, there are also cases where a company legitimately cannot
200	disclose its plans early in the development process.  Companies with
201	experienced kernel developers may choose to proceed in an open-loop manner
202	on the assumption that they will be able to avoid serious integration
203	problems later.  For companies without that sort of in-house expertise, the
204	best option is often to hire an outside developer to review the plans under
205	a non-disclosure agreement.  The Linux Foundation operates an NDA program
206	designed to help with this sort of situation; more information can be found
207	at:
209	    http://www.linuxfoundation.org/en/NDA_program
211	This kind of review is often enough to avoid serious problems later on
212	without requiring public disclosure of the project.
Hide Line Numbers
About Kernel Documentation Linux Kernel Contact Linux Resources Linux Blog

Information is copyright its respective author. All material is available from the Linux Kernel Source distributed under a GPL License. This page is provided as a free service by mjmwired.net.