Important Announcement: Please read https://www.redhat.com/archives/fedora-announce-list/2008-August/msg00008.html and https://www.redhat.com/archives/fedora-announce-list/2008-August/msg00009.html for information.
Extras/GetKernelModulesUpstream
From FedoraProject
It is still under construction and content may change. Do not rely on the information on this page.
Packages with kernel modules in Fedora Extras
Fedora Extras has a special rule for packages containing kernel-modules: "Ask the Fedora Extras Steering Committee (FESCo) for permission if the module is allowed for Extras. This requires that you give at least the following informations: [...] A publishable explanation from the author(s) why the module is not merged with the mainline kernel yet and when it's planned to get merged. "
This page tries to do it's best to explain why the above process was set in place.
The answer in short: There are a lot of political and technical reasons why it best for everyone if all linux kernel modules quickly become part of the upstream kernel, the kernel tree managed by Linus Torvalds and available from kernel.org (also known as linus-kernel or vanilla kernel). It's quite hard to differentiate what's political and what's technical. This page will simply lay down all those reasons the authors thought where important for this discussion.
There are some Fedora specific reasons, but first the more important general ones:
- a module that was included in linus-kernel has undergone a lot of QA -- especially before being included but a lot of eyes also watch it later. This is a good thing, but module authors often fear the initial review process for no good reasons (read: they often don't know yet that going through the initial QA with a incomplete but small drivers is much easier for everyone than to review a driver after it was developed outside the kernels scope for months or even years).
- stuff in the kernels get modified quite often -- API changes, modification that take away an include file, module exports and stuff like that often breaks external modules. This applies not just to large rebases (ie, 2.6.17->2.6.18), but even within the same version stream. Modules that are included in linus-kernel get fixed automatically when the kernel-developers adjust the API the module uses. The Linux kernel has no stable driver API, so this is a huge advantage for the module author because they would otherwise have to adapt their driver for every API change. The API changes are rarely highlighted from one kernel release to another, so they often have to be discovered and debugged after a release has come out if the module is out of tree.
- a module that gets included into linus-kernel will be included in practically all Linux distributions. And it will get installed by default normally and therefore will be automatically available when you need it. Imagine you are traveling and you forgot your WLAN-Card. A happy co-worker lends you his -- if the module is part of the kernel packages it will be around and the card will "play" after plugging. If the drivers is packaged separately you'er out of luck because you often won't be able to install it because you don't have the distribution's media at hand and no network access.
- sometimes kernel modules depend on other parts of the kernel that need to be updated to make the modules work properly. This can break other drivers that are already part of the kernel or (even worse) other drivers that yet again require other changes to the subsystem (read: great mess). The ipw3945 drivers for example required a newer ieee80211-Subsystem until recently; but by installing that you might break the modules that were build together with the kernel against its ieee80211-Subsystem .
First all those reasons that are specific to Fedora:
- why should a kernel module be good enough for Fedora if it's not good enough for the QA on lkml / for linus-kernel?
- rebuilding a kernel-modules takes time. Let's say rebuilding a single, imaginable module as part of the kernel might take 15 seconds per kernel. Building the same modules as kmod in Extras takes a lot more time because the buildroots have to be set up anew for each arch (i586, i686, x86_64, ppc, ppc64) and each kernel-module. That takes a lot of time -- depending on how fast the buildsys are, it can take about 2 minutes and up to 5 minutes per arch(or even more). That doesn't sound much for one kernel module package, but it becomes quiet a bunch of time when you have to rebuild a big number of kmod packages.
- Currently most of the the Extras contributors are more packagers that developers. Most have no or only limited programming skills. So if the API in the kernel changes and the modules needs adjustments they have to wait until the upstream-authors of the kernel modules makes necessary changes. That can take time -- therefore the user might run into a situation where he's unable to use all the modules he needs. The scenarios below will show some examples of this problem. Start situations for both scenarios: User runs kernel 2.6.15 and needs kmod-foo and kmod-bar for his hardware. Both that are packaged in Extras. Now kernel 2.6.16 is shipped as official update by Fedora Core.
- 2.6.16 is fixes a serious security flaw; yum pulls in the new kernel quickly, but the kmods are not built yet so he still uses the old kernel until the new modules get published. In the meantime the system can become a victim of a bad Internet worm.
- 2.6.16 and kmod-foo get installed, but kmod-bar doesn't build properly for 2.6.16. Users sticks to 2.6.15. An update foo-common get build in the repo, but it only works on 2.6.16 and later. User still tries to stick to 2.6.15, but now suddenly runs into a big data-corruption problem with kmod-bar that is fixed in the new version for the new kernel only. But he can't update to it because he also needs kmod-foo, which still doesn't build for 2.6.16.
- kernel-modules that are not part of the kernel will lead to "not my bug scenarios" where the user is left alone in the cold with problems he detects. Verbose: A kernel-module can do a lot of good or bad stuff in the memory. So a module with a bug can corrupt data in other areas of the kernel. This can lead to problems that show up in areas of the kernel were you wouldn't expect them because they have nothing to do with the kernel module at first sight. The developers of the kernel from Fedora Core therefor refuse to debug kernels where modules that are not part of the main kernel from Fedora Core are loaded. The maintainer of a kmod in Fedora Extras on the other hand often simply hasn't the knowledge to debug or fix problems that might be related to his module. This is yet another reason why proper review from the upstream kernel is so important.
- The kernel module packaging in Fedora Extras with rpm-packages is a compromise -- it works quite well, but none of the schemes the people that created the kmod-packaging standard looked at worked perfectly in rpm, yum and all the other packaging tools. So users might run into some update issues now and then.
Note: some people will now say "Well, some of the problems around packaging kernel-modules in Fedora Extras can be fixed by including the kernel-modules as part of the basic kernel package." That's true, but it will create other problems. The main one: the core kernel-developer would have a bunch of more work to fix modules that are not compatible with new versions of the upstream kernel (the modules can't simply be dropped, because that would leave users out in the cold or with broken systems). We had those problems in the days of RHL/FC with kernel 2.4 -- nobody wants this PITA back.
Another note: Fedora Extras tries to generally make sure that modules get merged upstream; but there are special circumstances now and then where FESCo can and will allow modules that never will get upstream (e.g. maybe gfs1 for compatible reasons while in fact gfs2 gets merged upstream). FESCo will decide on a case-by-case-basis.
Links
- Documentation/stable_api_nonsense.txt - why Linux does not have a binary kernel interface, nor does it have a stable kernel interface.
- Greg KH's OLS 2006 keynote - contains some reasons why it's best for everyone that kernel-modules get part of the linus-kernel
