From Fedora Project Wiki
(add more to the sources section)
Line 55: Line 55:
 
===== Individual patches =====
 
===== Individual patches =====
  
Individual patches are listed in the kernel.spec file in the standalone patches section.  These are typically used for bugfixes, however we have a handful of patches we carry for various addons or features.  Typically, a new patch is added by searching for the <code># END OF PATCH DEFINITIONS</code> string and adding a new entry above it such as:
+
Individual patches are listed in the kernel.spec file in the standalone patches section.  These are typically used for bugfixes, however we have a handful of patches we carry for various addons or features.  Patches must have a strip level of 1.
 +
 
 +
Typically, a new patch is added by searching for the <code># END OF PATCH DEFINITIONS</code> string and adding a new entry above it such as:
  
 
<pre>
 
<pre>
Line 80: Line 82:
  
 
following the same comment format/rules as the definition section.  Once those two entries are added to the spec file, the patch application should be visible in the output of the <code>fedpkg prep</code> command.
 
following the same comment format/rules as the definition section.  Once those two entries are added to the spec file, the patch application should be visible in the output of the <code>fedpkg prep</code> command.
 +
 +
=== Functions ===
 +
There are a few functions we use to build the kernel.  This section will cover those.
 +
 +
===== ApplyPatch =====
 +
As noted in the above section, the ApplyPatch function is called to actually apply a patch.  This is used for both large patches (e.g. patch-4.1-rc2.xz) and standalone patches.  The function verifies that the patch being applied is also defined in the spec file, transparently handles any compression, and then applies the patch via:
 +
 +
<code> patch -p1 -F1 s </code>
 +
 +
There is a similar ApplyOptionalPatch function that will optionally apply a patch if it is not empty and skip it otherwise.  This is seldom used.
 +
 +
===== BuildKernel =====
 +
This is the function that is primarily responsible for actually building the kernel and modules.  It takes 'MakeTarget' 'KernelImage' 'Flavour' and optionally 'InstallName' arguments.
 +
 +
''MakeTarget'': This is the name of the vmlinux target to build.  This can vary among the architectures.  On x86, it is bzImage.  On powerpc it is vmlinux.  The architectures define this in a make_target macro elsewhere in the spec.
 +
 +
''KernelImage'': This is the relative path to the final 'MakeTarget' binary that is produced during the build.  E.g. it is arch/x86_64/boot/bzIMage on x86_64.  This is similarly defined via a kernel_image macro elsewhere.
 +
 +
''Flavour'': This is an optional argument the allows us to build multiple kernel flavors for each rpmbuild invocation.  E.g. we will be both a normal and a debug kernel, or on i686 we build i686, i686+PAE, i686+debug, i686+PAEdebug.
 +
 +
''InstallName'': This is optional and not actually specified anywhere today.  The default value of 'vmlinuz' is used.  It is the leading portion of the name of the final kernel binary that we install.
 +
 +
The BuildKernel function takes these arguments and builds the kernel and modules via the kernel's makefile system.  It generates the correct .config for the architecture, calls make $MakeTarget and make modules, installs the vmlinux, modules, and vdso binaries in the appropriate paths.  It also sets up the proper headers and paths for the kernel-devel and debuginfo packages.  Essentially this is where virtually all of the build and installation happens for every rpmbuild.
 +
 +
=== Other ===
 +
There are a few other sections in the spec file that control various items.
 +
 +
''perf'': The perf command is build after all of the various kernel flavors are built.  This is done via an invocation to the perf make system using the perf_make macro to control various options.
 +
 +
''Other tools'': Various other tools that are contained within the kernel sources are also built after perf.
 +
 +
''Signing modules'': Due to the way that rpmbuild and debuginfo interact with binaries, we must sign the modules at a particular time during rpmbuild or it will strip off the signatures of the modules. This is done by creating a __modsign_install_post macro and calling that from an overidden __spec_install_post definition.  You shouldn't need to modify this.
 +
 +
=== Package definitions ===
 +
 +
The kernel is shipped with items split into various packages and sub-packages.  These are listed below.  Unless you are modifying how the kernel is actually packaged within Fedora, you shouldn't need to modify any of this.
 +
 +
'''kernel''': This is a meta-package the Requires kernel-core and kernel-modules
 +
 +
'''kernel-core''': This is the main vmlinux binary, and a handful of other "core" modules.  Essentially this is all that is required to boot a Fedora cloud image or a small KVM guest machine.
 +
 +
'''kernel-modules''': This package contains the remaining modules that are not in kernel-core that are normally used on real hardware.  Things like wireless and ethernet drivers, disk drivers, filesystems, etc.  On a Fedora Server, Workstation, or any other install that is intended to work on real hardware, this is required.
 +
 +
'''kernel-modules-extra''': This is a package that contains seldom used modules.  It is optional and not included in a default install.
 +
 +
'''kernel-devel''': This is the package that provides the appropriate symlinks and header to build out-of-tree modules against the Fedora kernel.
 +
 +
'''kernel-tools''': This contains various tools found in the kernel sources
 +
 +
'''kernel-tools-libs''': This contains various libraries the above tools provide/use
 +
 +
'''perf''': This provides the perf command and modules.
 +
 +
'''*-debuginfo*''': This provides the stripped out debug information for all the kernels, modules, and tools.  It is giant and unless you're doing kernel development and need to load it in gdb to get offsets or backtraces, or better oops information, you do not need it installed.

Revision as of 17:57, 14 April 2015

The Fedora kernel.spec

Like all RPMs, the Fedora kernel starts with a spec file and a set of tarballs and patches. This page will describe some of the sections of the kernel.spec file, what they're used for, and how to modify them.

Macros

There are a few important macros that we modify often. These are listed below.

%global released_kernel

The released_kernel macro controls whether the rest of the macros and functions are dealing with an upstream released final kernel version. In the stable Fedora branches, this should almost always be set to 1. This will tell the other macros to use the final tarball (e.g. linux-4.0.tar.xz) and base_sublevel as the basis for the version numbers. In rawhide, released_kernel will often be set to 0 as we often ship pre-release git snapshots and -rcX kernels. Having it set to 0 will cause the proper versioning to happen for these non-released kernels.

%define base_sublevel

The base_sublevel macro sets up the portion of the versioning after the major version number. E.g. base_sublevel of 0 will be 4.0. If it is set to 1, the versioning will be 1. Note, it is a reflection of the actual tarball we are building on top of, so if released_kernel is 0, base_sublevel will still be using the previous version. E.g. 4.1-rc1.git1 starts with the 4.0 tarball, so base_sublevel should be 0.

%define stable_update

The stable_update macro controls the .y portion of the versioning and is responsible for applying the upstream stable release patches. If it is set to 0, no stable patch will be applied. If it is set to another number, it will apply that .y version number on top of the base_sublevel tarball. E.g. stable_update 2 and base_sublevel 0 would apply patch-4.0.2.xz on top of linux-4.0.tar.xz. This macro is only in use when released_kernel is 1.

%define rcrev

When released_kernel is 0, we are building RC or merge window kernels. The rcrev macro contains the numeric value of the upstream RC release for the kernel we are targetting. Merge window kernels will have an rcrev of 0. All other -rcX releases will match their upstream release, e.g. 4.1-rc2 would have an rcrev value of 2. This macro is not used when released_kernel is 1.

%define gitrev

When we build a git snapshot kernel, the gitrev macro contains the value of the git snapshot patch to apply. E.g. 4.1-rc2-git3 would have a gitrev value of 3. This will cause the spec to apply patch-4.1-rc2-git3.xz on top of linux-4.0.xz+patch-4.1-rc2.xz. This is not used when released_kernel is 0.

Note: We have a script that generated the git snapshot patches and will modify this value in the spec accordingly. Manual modification of this macro does not happen often.

%global baserelease

The baserelease macro controls the final number in the RPM Release field. E.g a 4.0.0-1.fc23 kernel has a baserelease value of 1. 4.1.0-0.rc2.git1.3.fc23 has a baserelease value of 3. This macro is the most often edited macro we have and should be modified for every commit we do. When we rebase to a new kernel, we reset it back to the value appropriate for that branch. To help with upgrade paths, each Fedora branch has a different starting value for baserelease. These are listed below.

* rawhide/Branched: Starts at 1 and increments from there.
* Most recent stable release N: Starts at 300 and increments from there
* N-1 release: Starts at 200 and increments from there
* N-2 release: Starts at 100 and increments from there

This way, the oldest release always has the "oldest" Name-Version-Release combination for that upstream kernel release.

There are other macros that control some other things, however we'll skip those for now and focus on the main macros used for build output.

Sources

Tarballs and large patch files

The sources file lists the tarballs and large patch files that we used to explode the kernel sources. It will contain the file name and a hash value for that file. The contents will look similar to:

a86916bd12798220da9eb4a1eec3616d  linux-4.0.tar.xz
d125eecce68ab6fb5f1f23523c2c04b8  perf-man-4.0.tar.gz

The perf-man tarball is a self created tarball of the perf man pages. This is generated once per upstream kernel release. As -rcX patches are released and git snapshots are generated, we also add those via the sources file. This is done with the fedpkg upload 'filename' command.

Individual patches

Individual patches are listed in the kernel.spec file in the standalone patches section. These are typically used for bugfixes, however we have a handful of patches we carry for various addons or features. Patches must have a strip level of 1.

Typically, a new patch is added by searching for the # END OF PATCH DEFINITIONS string and adding a new entry above it such as:

#rhbz 1196825
Patch26140: security-yama-Remove-unnecessary-selects-from-Kconfi.patch

Here we list the Red Hat bugzilla number as a comment above the PatchXXXXX: 'filename' listing. The only stipulation here is that the XXXXX number has to be unique in the spec file. Normally we just increment from the file above it. For security fixes, we tend to use this format:

#CVE-2015-2150 rhbz 1196266 1200397
Patch26175: xen-pciback-Don-t-disable-PCI_COMMAND-on-PCI-device-.patch

which lists the CVE number, and the two associated Bugzilla numbers (the overall CVE tracking bug, and the Fedora specific instance).

Actually applying individual patches

Listing the patch as a definition as describe above is required, but does not automatically get the patch applied to the kernel source tree. To do that, the patch must also be listed in the patch application section. Similar to above, typically this is done by searching for the # END OF PATCH APPLICATIONS string and listing the patch as:

#rhbz 1201532
ApplyPatch HID-multitouch-add-support-of-clickpads.patch

following the same comment format/rules as the definition section. Once those two entries are added to the spec file, the patch application should be visible in the output of the fedpkg prep command.

Functions

There are a few functions we use to build the kernel. This section will cover those.

ApplyPatch

As noted in the above section, the ApplyPatch function is called to actually apply a patch. This is used for both large patches (e.g. patch-4.1-rc2.xz) and standalone patches. The function verifies that the patch being applied is also defined in the spec file, transparently handles any compression, and then applies the patch via:

patch -p1 -F1 s

There is a similar ApplyOptionalPatch function that will optionally apply a patch if it is not empty and skip it otherwise. This is seldom used.

BuildKernel

This is the function that is primarily responsible for actually building the kernel and modules. It takes 'MakeTarget' 'KernelImage' 'Flavour' and optionally 'InstallName' arguments.

MakeTarget: This is the name of the vmlinux target to build. This can vary among the architectures. On x86, it is bzImage. On powerpc it is vmlinux. The architectures define this in a make_target macro elsewhere in the spec.

KernelImage: This is the relative path to the final 'MakeTarget' binary that is produced during the build. E.g. it is arch/x86_64/boot/bzIMage on x86_64. This is similarly defined via a kernel_image macro elsewhere.

Flavour: This is an optional argument the allows us to build multiple kernel flavors for each rpmbuild invocation. E.g. we will be both a normal and a debug kernel, or on i686 we build i686, i686+PAE, i686+debug, i686+PAEdebug.

InstallName: This is optional and not actually specified anywhere today. The default value of 'vmlinuz' is used. It is the leading portion of the name of the final kernel binary that we install.

The BuildKernel function takes these arguments and builds the kernel and modules via the kernel's makefile system. It generates the correct .config for the architecture, calls make $MakeTarget and make modules, installs the vmlinux, modules, and vdso binaries in the appropriate paths. It also sets up the proper headers and paths for the kernel-devel and debuginfo packages. Essentially this is where virtually all of the build and installation happens for every rpmbuild.

Other

There are a few other sections in the spec file that control various items.

perf: The perf command is build after all of the various kernel flavors are built. This is done via an invocation to the perf make system using the perf_make macro to control various options.

Other tools: Various other tools that are contained within the kernel sources are also built after perf.

Signing modules: Due to the way that rpmbuild and debuginfo interact with binaries, we must sign the modules at a particular time during rpmbuild or it will strip off the signatures of the modules. This is done by creating a __modsign_install_post macro and calling that from an overidden __spec_install_post definition. You shouldn't need to modify this.

Package definitions

The kernel is shipped with items split into various packages and sub-packages. These are listed below. Unless you are modifying how the kernel is actually packaged within Fedora, you shouldn't need to modify any of this.

kernel: This is a meta-package the Requires kernel-core and kernel-modules

kernel-core: This is the main vmlinux binary, and a handful of other "core" modules. Essentially this is all that is required to boot a Fedora cloud image or a small KVM guest machine.

kernel-modules: This package contains the remaining modules that are not in kernel-core that are normally used on real hardware. Things like wireless and ethernet drivers, disk drivers, filesystems, etc. On a Fedora Server, Workstation, or any other install that is intended to work on real hardware, this is required.

kernel-modules-extra: This is a package that contains seldom used modules. It is optional and not included in a default install.

kernel-devel: This is the package that provides the appropriate symlinks and header to build out-of-tree modules against the Fedora kernel.

kernel-tools: This contains various tools found in the kernel sources

kernel-tools-libs: This contains various libraries the above tools provide/use

perf: This provides the perf command and modules.

*-debuginfo*: This provides the stripped out debug information for all the kernels, modules, and tools. It is giant and unless you're doing kernel development and need to load it in gdb to get offsets or backtraces, or better oops information, you do not need it installed.