Based on kernel version 4.16.1. Page generated on 2018-04-09 11:52 EST.
1 This document provides information for the BPF subsystem about various 2 workflows related to reporting bugs, submitting patches, and queueing 3 patches for stable kernels. 4 5 For general information about submitting patches, please refer to 6 Documentation/process/. This document only describes additional specifics 7 related to BPF. 8 9 Reporting bugs: 10 --------------- 11 12 Q: How do I report bugs for BPF kernel code? 13 14 A: Since all BPF kernel development as well as bpftool and iproute2 BPF 15 loader development happens through the netdev kernel mailing list, 16 please report any found issues around BPF to the following mailing 17 list: 18 19 netdev@vger.kernel.org 20 21 This may also include issues related to XDP, BPF tracing, etc. 22 23 Given netdev has a high volume of traffic, please also add the BPF 24 maintainers to Cc (from kernel MAINTAINERS file): 25 26 Alexei Starovoitov <ast@kernel.org> 27 Daniel Borkmann <daniel@iogearbox.net> 28 29 In case a buggy commit has already been identified, make sure to keep 30 the actual commit authors in Cc as well for the report. They can 31 typically be identified through the kernel's git tree. 32 33 Please do *not* report BPF issues to bugzilla.kernel.org since it 34 is a guarantee that the reported issue will be overlooked. 35 36 Submitting patches: 37 ------------------- 38 39 Q: To which mailing list do I need to submit my BPF patches? 40 41 A: Please submit your BPF patches to the netdev kernel mailing list: 42 43 netdev@vger.kernel.org 44 45 Historically, BPF came out of networking and has always been maintained 46 by the kernel networking community. Although these days BPF touches 47 many other subsystems as well, the patches are still routed mainly 48 through the networking community. 49 50 In case your patch has changes in various different subsystems (e.g. 51 tracing, security, etc), make sure to Cc the related kernel mailing 52 lists and maintainers from there as well, so they are able to review 53 the changes and provide their Acked-by's to the patches. 54 55 Q: Where can I find patches currently under discussion for BPF subsystem? 56 57 A: All patches that are Cc'ed to netdev are queued for review under netdev 58 patchwork project: 59 60 http://patchwork.ozlabs.org/project/netdev/list/ 61 62 Those patches which target BPF, are assigned to a 'bpf' delegate for 63 further processing from BPF maintainers. The current queue with 64 patches under review can be found at: 65 66 https://patchwork.ozlabs.org/project/netdev/list/?delegate=77147 67 68 Once the patches have been reviewed by the BPF community as a whole 69 and approved by the BPF maintainers, their status in patchwork will be 70 changed to 'Accepted' and the submitter will be notified by mail. This 71 means that the patches look good from a BPF perspective and have been 72 applied to one of the two BPF kernel trees. 73 74 In case feedback from the community requires a respin of the patches, 75 their status in patchwork will be set to 'Changes Requested', and purged 76 from the current review queue. Likewise for cases where patches would 77 get rejected or are not applicable to the BPF trees (but assigned to 78 the 'bpf' delegate). 79 80 Q: How do the changes make their way into Linux? 81 82 A: There are two BPF kernel trees (git repositories). Once patches have 83 been accepted by the BPF maintainers, they will be applied to one 84 of the two BPF trees: 85 86 https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/ 87 https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git/ 88 89 The bpf tree itself is for fixes only, whereas bpf-next for features, 90 cleanups or other kind of improvements ("next-like" content). This is 91 analogous to net and net-next trees for networking. Both bpf and 92 bpf-next will only have a master branch in order to simplify against 93 which branch patches should get rebased to. 94 95 Accumulated BPF patches in the bpf tree will regularly get pulled 96 into the net kernel tree. Likewise, accumulated BPF patches accepted 97 into the bpf-next tree will make their way into net-next tree. net and 98 net-next are both run by David S. Miller. From there, they will go 99 into the kernel mainline tree run by Linus Torvalds. To read up on the 100 process of net and net-next being merged into the mainline tree, see 101 the netdev FAQ under: 102 103 Documentation/networking/netdev-FAQ.txt 104 105 Occasionally, to prevent merge conflicts, we might send pull requests 106 to other trees (e.g. tracing) with a small subset of the patches, but 107 net and net-next are always the main trees targeted for integration. 108 109 The pull requests will contain a high-level summary of the accumulated 110 patches and can be searched on netdev kernel mailing list through the 111 following subject lines (yyyy-mm-dd is the date of the pull request): 112 113 pull-request: bpf yyyy-mm-dd 114 pull-request: bpf-next yyyy-mm-dd 115 116 Q: How do I indicate which tree (bpf vs. bpf-next) my patch should be 117 applied to? 118 119 A: The process is the very same as described in the netdev FAQ, so 120 please read up on it. The subject line must indicate whether the 121 patch is a fix or rather "next-like" content in order to let the 122 maintainers know whether it is targeted at bpf or bpf-next. 123 124 For fixes eventually landing in bpf -> net tree, the subject must 125 look like: 126 127 git format-patch --subject-prefix='PATCH bpf' start..finish 128 129 For features/improvements/etc that should eventually land in 130 bpf-next -> net-next, the subject must look like: 131 132 git format-patch --subject-prefix='PATCH bpf-next' start..finish 133 134 If unsure whether the patch or patch series should go into bpf 135 or net directly, or bpf-next or net-next directly, it is not a 136 problem either if the subject line says net or net-next as target. 137 It is eventually up to the maintainers to do the delegation of 138 the patches. 139 140 If it is clear that patches should go into bpf or bpf-next tree, 141 please make sure to rebase the patches against those trees in 142 order to reduce potential conflicts. 143 144 In case the patch or patch series has to be reworked and sent out 145 again in a second or later revision, it is also required to add a 146 version number (v2, v3, ...) into the subject prefix: 147 148 git format-patch --subject-prefix='PATCH net-next v2' start..finish 149 150 When changes have been requested to the patch series, always send the 151 whole patch series again with the feedback incorporated (never send 152 individual diffs on top of the old series). 153 154 Q: What does it mean when a patch gets applied to bpf or bpf-next tree? 155 156 A: It means that the patch looks good for mainline inclusion from 157 a BPF point of view. 158 159 Be aware that this is not a final verdict that the patch will 160 automatically get accepted into net or net-next trees eventually: 161 162 On the netdev kernel mailing list reviews can come in at any point 163 in time. If discussions around a patch conclude that they cannot 164 get included as-is, we will either apply a follow-up fix or drop 165 them from the trees entirely. Therefore, we also reserve to rebase 166 the trees when deemed necessary. After all, the purpose of the tree 167 is to i) accumulate and stage BPF patches for integration into trees 168 like net and net-next, and ii) run extensive BPF test suite and 169 workloads on the patches before they make their way any further. 170 171 Once the BPF pull request was accepted by David S. Miller, then 172 the patches end up in net or net-next tree, respectively, and 173 make their way from there further into mainline. Again, see the 174 netdev FAQ for additional information e.g. on how often they are 175 merged to mainline. 176 177 Q: How long do I need to wait for feedback on my BPF patches? 178 179 A: We try to keep the latency low. The usual time to feedback will 180 be around 2 or 3 business days. It may vary depending on the 181 complexity of changes and current patch load. 182 183 Q: How often do you send pull requests to major kernel trees like 184 net or net-next? 185 186 A: Pull requests will be sent out rather often in order to not 187 accumulate too many patches in bpf or bpf-next. 188 189 As a rule of thumb, expect pull requests for each tree regularly 190 at the end of the week. In some cases pull requests could additionally 191 come also in the middle of the week depending on the current patch 192 load or urgency. 193 194 Q: Are patches applied to bpf-next when the merge window is open? 195 196 A: For the time when the merge window is open, bpf-next will not be 197 processed. This is roughly analogous to net-next patch processing, 198 so feel free to read up on the netdev FAQ about further details. 199 200 During those two weeks of merge window, we might ask you to resend 201 your patch series once bpf-next is open again. Once Linus released 202 a v*-rc1 after the merge window, we continue processing of bpf-next. 203 204 For non-subscribers to kernel mailing lists, there is also a status 205 page run by David S. Miller on net-next that provides guidance: 206 207 http://vger.kernel.org/~davem/net-next.html 208 209 Q: I made a BPF verifier change, do I need to add test cases for 210 BPF kernel selftests? 211 212 A: If the patch has changes to the behavior of the verifier, then yes, 213 it is absolutely necessary to add test cases to the BPF kernel 214 selftests suite. If they are not present and we think they are 215 needed, then we might ask for them before accepting any changes. 216 217 In particular, test_verifier.c is tracking a high number of BPF test 218 cases, including a lot of corner cases that LLVM BPF back end may 219 generate out of the restricted C code. Thus, adding test cases is 220 absolutely crucial to make sure future changes do not accidentally 221 affect prior use-cases. Thus, treat those test cases as: verifier 222 behavior that is not tracked in test_verifier.c could potentially 223 be subject to change. 224 225 Q: When should I add code to samples/bpf/ and when to BPF kernel 226 selftests? 227 228 A: In general, we prefer additions to BPF kernel selftests rather than 229 samples/bpf/. The rationale is very simple: kernel selftests are 230 regularly run by various bots to test for kernel regressions. 231 232 The more test cases we add to BPF selftests, the better the coverage 233 and the less likely it is that those could accidentally break. It is 234 not that BPF kernel selftests cannot demo how a specific feature can 235 be used. 236 237 That said, samples/bpf/ may be a good place for people to get started, 238 so it might be advisable that simple demos of features could go into 239 samples/bpf/, but advanced functional and corner-case testing rather 240 into kernel selftests. 241 242 If your sample looks like a test case, then go for BPF kernel selftests 243 instead! 244 245 Q: When should I add code to the bpftool? 246 247 A: The main purpose of bpftool (under tools/bpf/bpftool/) is to provide 248 a central user space tool for debugging and introspection of BPF programs 249 and maps that are active in the kernel. If UAPI changes related to BPF 250 enable for dumping additional information of programs or maps, then 251 bpftool should be extended as well to support dumping them. 252 253 Q: When should I add code to iproute2's BPF loader? 254 255 A: For UAPI changes related to the XDP or tc layer (e.g. cls_bpf), the 256 convention is that those control-path related changes are added to 257 iproute2's BPF loader as well from user space side. This is not only 258 useful to have UAPI changes properly designed to be usable, but also 259 to make those changes available to a wider user base of major 260 downstream distributions. 261 262 Q: Do you accept patches as well for iproute2's BPF loader? 263 264 A: Patches for the iproute2's BPF loader have to be sent to: 265 266 netdev@vger.kernel.org 267 268 While those patches are not processed by the BPF kernel maintainers, 269 please keep them in Cc as well, so they can be reviewed. 270 271 The official git repository for iproute2 is run by Stephen Hemminger 272 and can be found at: 273 274 https://git.kernel.org/pub/scm/linux/kernel/git/shemminger/iproute2.git/ 275 276 The patches need to have a subject prefix of '[PATCH iproute2 master]' 277 or '[PATCH iproute2 net-next]'. 'master' or 'net-next' describes the 278 target branch where the patch should be applied to. Meaning, if kernel 279 changes went into the net-next kernel tree, then the related iproute2 280 changes need to go into the iproute2 net-next branch, otherwise they 281 can be targeted at master branch. The iproute2 net-next branch will get 282 merged into the master branch after the current iproute2 version from 283 master has been released. 284 285 Like BPF, the patches end up in patchwork under the netdev project and 286 are delegated to 'shemminger' for further processing: 287 288 http://patchwork.ozlabs.org/project/netdev/list/?delegate=389 289 290 Q: What is the minimum requirement before I submit my BPF patches? 291 292 A: When submitting patches, always take the time and properly test your 293 patches *prior* to submission. Never rush them! If maintainers find 294 that your patches have not been properly tested, it is a good way to 295 get them grumpy. Testing patch submissions is a hard requirement! 296 297 Note, fixes that go to bpf tree *must* have a Fixes: tag included. The 298 same applies to fixes that target bpf-next, where the affected commit 299 is in net-next (or in some cases bpf-next). The Fixes: tag is crucial 300 in order to identify follow-up commits and tremendously helps for people 301 having to do backporting, so it is a must have! 302 303 We also don't accept patches with an empty commit message. Take your 304 time and properly write up a high quality commit message, it is 305 essential! 306 307 Think about it this way: other developers looking at your code a month 308 from now need to understand *why* a certain change has been done that 309 way, and whether there have been flaws in the analysis or assumptions 310 that the original author did. Thus providing a proper rationale and 311 describing the use-case for the changes is a must. 312 313 Patch submissions with >1 patch must have a cover letter which includes 314 a high level description of the series. This high level summary will 315 then be placed into the merge commit by the BPF maintainers such that 316 it is also accessible from the git log for future reference. 317 318 Q: What do I need to consider when adding a new instruction or feature 319 that would require BPF JIT and/or LLVM integration as well? 320 321 A: We try hard to keep all BPF JITs up to date such that the same user 322 experience can be guaranteed when running BPF programs on different 323 architectures without having the program punt to the less efficient 324 interpreter in case the in-kernel BPF JIT is enabled. 325 326 If you are unable to implement or test the required JIT changes for 327 certain architectures, please work together with the related BPF JIT 328 developers in order to get the feature implemented in a timely manner. 329 Please refer to the git log (arch/*/net/) to locate the necessary 330 people for helping out. 331 332 Also always make sure to add BPF test cases (e.g. test_bpf.c and 333 test_verifier.c) for new instructions, so that they can receive 334 broad test coverage and help run-time testing the various BPF JITs. 335 336 In case of new BPF instructions, once the changes have been accepted 337 into the Linux kernel, please implement support into LLVM's BPF back 338 end. See LLVM section below for further information. 339 340 Stable submission: 341 ------------------ 342 343 Q: I need a specific BPF commit in stable kernels. What should I do? 344 345 A: In case you need a specific fix in stable kernels, first check whether 346 the commit has already been applied in the related linux-*.y branches: 347 348 https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git/ 349 350 If not the case, then drop an email to the BPF maintainers with the 351 netdev kernel mailing list in Cc and ask for the fix to be queued up: 352 353 netdev@vger.kernel.org 354 355 The process in general is the same as on netdev itself, see also the 356 netdev FAQ document. 357 358 Q: Do you also backport to kernels not currently maintained as stable? 359 360 A: No. If you need a specific BPF commit in kernels that are currently not 361 maintained by the stable maintainers, then you are on your own. 362 363 The current stable and longterm stable kernels are all listed here: 364 365 https://www.kernel.org/ 366 367 Q: The BPF patch I am about to submit needs to go to stable as well. What 368 should I do? 369 370 A: The same rules apply as with netdev patch submissions in general, see 371 netdev FAQ under: 372 373 Documentation/networking/netdev-FAQ.txt 374 375 Never add "Cc: stable@vger.kernel.org" to the patch description, but 376 ask the BPF maintainers to queue the patches instead. This can be done 377 with a note, for example, under the "---" part of the patch which does 378 not go into the git log. Alternatively, this can be done as a simple 379 request by mail instead. 380 381 Q: Where do I find currently queued BPF patches that will be submitted 382 to stable? 383 384 A: Once patches that fix critical bugs got applied into the bpf tree, they 385 are queued up for stable submission under: 386 387 http://patchwork.ozlabs.org/bundle/bpf/stable/?state=* 388 389 They will be on hold there at minimum until the related commit made its 390 way into the mainline kernel tree. 391 392 After having been under broader exposure, the queued patches will be 393 submitted by the BPF maintainers to the stable maintainers. 394 395 Testing patches: 396 ---------------- 397 398 Q: Which BPF kernel selftests version should I run my kernel against? 399 400 A: If you run a kernel xyz, then always run the BPF kernel selftests from 401 that kernel xyz as well. Do not expect that the BPF selftest from the 402 latest mainline tree will pass all the time. 403 404 In particular, test_bpf.c and test_verifier.c have a large number of 405 test cases and are constantly updated with new BPF test sequences, or 406 existing ones are adapted to verifier changes e.g. due to verifier 407 becoming smarter and being able to better track certain things. 408 409 LLVM: 410 ----- 411 412 Q: Where do I find LLVM with BPF support? 413 414 A: The BPF back end for LLVM is upstream in LLVM since version 3.7.1. 415 416 All major distributions these days ship LLVM with BPF back end enabled, 417 so for the majority of use-cases it is not required to compile LLVM by 418 hand anymore, just install the distribution provided package. 419 420 LLVM's static compiler lists the supported targets through 'llc --version', 421 make sure BPF targets are listed. Example: 422 423 $ llc --version 424 LLVM (http://llvm.org/): 425 LLVM version 6.0.0svn 426 Optimized build. 427 Default target: x86_64-unknown-linux-gnu 428 Host CPU: skylake 429 430 Registered Targets: 431 bpf - BPF (host endian) 432 bpfeb - BPF (big endian) 433 bpfel - BPF (little endian) 434 x86 - 32-bit X86: Pentium-Pro and above 435 x86-64 - 64-bit X86: EM64T and AMD64 436 437 For developers in order to utilize the latest features added to LLVM's 438 BPF back end, it is advisable to run the latest LLVM releases. Support 439 for new BPF kernel features such as additions to the BPF instruction 440 set are often developed together. 441 442 All LLVM releases can be found at: http://releases.llvm.org/ 443 444 Q: Got it, so how do I build LLVM manually anyway? 445 446 A: You need cmake and gcc-c++ as build requisites for LLVM. Once you have 447 that set up, proceed with building the latest LLVM and clang version 448 from the git repositories: 449 450 $ git clone http://llvm.org/git/llvm.git 451 $ cd llvm/tools 452 $ git clone --depth 1 http://llvm.org/git/clang.git 453 $ cd ..; mkdir build; cd build 454 $ cmake .. -DLLVM_TARGETS_TO_BUILD="BPF;X86" \ 455 -DBUILD_SHARED_LIBS=OFF \ 456 -DCMAKE_BUILD_TYPE=Release \ 457 -DLLVM_BUILD_RUNTIME=OFF 458 $ make -j $(getconf _NPROCESSORS_ONLN) 459 460 The built binaries can then be found in the build/bin/ directory, where 461 you can point the PATH variable to. 462 463 Q: Should I notify BPF kernel maintainers about issues in LLVM's BPF code 464 generation back end or about LLVM generated code that the verifier 465 refuses to accept? 466 467 A: Yes, please do! LLVM's BPF back end is a key piece of the whole BPF 468 infrastructure and it ties deeply into verification of programs from the 469 kernel side. Therefore, any issues on either side need to be investigated 470 and fixed whenever necessary. 471 472 Therefore, please make sure to bring them up at netdev kernel mailing 473 list and Cc BPF maintainers for LLVM and kernel bits: 474 475 Yonghong Song <yhs@fb.com> 476 Alexei Starovoitov <ast@kernel.org> 477 Daniel Borkmann <daniel@iogearbox.net> 478 479 LLVM also has an issue tracker where BPF related bugs can be found: 480 481 https://bugs.llvm.org/buglist.cgi?quicksearch=bpf 482 483 However, it is better to reach out through mailing lists with having 484 maintainers in Cc. 485 486 Q: I have added a new BPF instruction to the kernel, how can I integrate 487 it into LLVM? 488 489 A: LLVM has a -mcpu selector for the BPF back end in order to allow the 490 selection of BPF instruction set extensions. By default the 'generic' 491 processor target is used, which is the base instruction set (v1) of BPF. 492 493 LLVM has an option to select -mcpu=probe where it will probe the host 494 kernel for supported BPF instruction set extensions and selects the 495 optimal set automatically. 496 497 For cross-compilation, a specific version can be select manually as well. 498 499 $ llc -march bpf -mcpu=help 500 Available CPUs for this target: 501 502 generic - Select the generic processor. 503 probe - Select the probe processor. 504 v1 - Select the v1 processor. 505 v2 - Select the v2 processor. 506 [...] 507 508 Newly added BPF instructions to the Linux kernel need to follow the same 509 scheme, bump the instruction set version and implement probing for the 510 extensions such that -mcpu=probe users can benefit from the optimization 511 transparently when upgrading their kernels. 512 513 If you are unable to implement support for the newly added BPF instruction 514 please reach out to BPF developers for help. 515 516 By the way, the BPF kernel selftests run with -mcpu=probe for better 517 test coverage. 518 519 Q: In some cases clang flag "-target bpf" is used but in other cases the 520 default clang target, which matches the underlying architecture, is used. 521 What is the difference and when I should use which? 522 523 A: Although LLVM IR generation and optimization try to stay architecture 524 independent, "-target <arch>" still has some impact on generated code: 525 526 - BPF program may recursively include header file(s) with file scope 527 inline assembly codes. The default target can handle this well, 528 while bpf target may fail if bpf backend assembler does not 529 understand these assembly codes, which is true in most cases. 530 531 - When compiled without -g, additional elf sections, e.g., 532 .eh_frame and .rela.eh_frame, may be present in the object file 533 with default target, but not with bpf target. 534 535 - The default target may turn a C switch statement into a switch table 536 lookup and jump operation. Since the switch table is placed 537 in the global readonly section, the bpf program will fail to load. 538 The bpf target does not support switch table optimization. 539 The clang option "-fno-jump-tables" can be used to disable 540 switch table generation. 541 542 You should use default target when: 543 544 - Your program includes a header file, e.g., ptrace.h, which eventually 545 pulls in some header files containing file scope host assembly codes. 546 - You can add "-fno-jump-tables" to work around the switch table issue. 547 548 Otherwise, you can use bpf target. 549 550 Happy BPF hacking!