Kernel Debugging Strategy
Fedora enables differing amounts of debugging in the kernel at various times depending on where we are in the release cycle.
What the various options do.
- DEBUG_LIST (debug linked list deletions)
- SPINLOCK_SLEEP (check if we're in code where we can sleep before using locks)
- DEBUG_SHIRQ (cause an interrupt to be generated as soon as we register an IRQ)
- DEBUG_RODATA (write protect read-only data, cause a pagefault if something tries to write to it)
- SLUB_DEBUG (perform a number of checks on allocated objects, poison free'd objects)
- DEBUG_HIGHMEM (Allow debugging of highmem issues on non-highmem boxes)
- DEBUG_MUTEXES=y DEBUG_RT_MUTEXES=y DEBUG_LOCK_ALLOC=y PROVE_LOCKING=y DEBUG_SPINLOCK (lock dependancy checker)
- DEBUG_VM=y (Various runtime checks in the VM code)
- DEBUG_PAGEALLOC (after freeing an object, unmap it from the address space. Attempts to access it cause an oops)
There are also a number of other DEBUG options, which just add extra printk's, or extra information in /proc or /sys, these are mostly uninteresting, and have little to no performance impact.
Release
'kernel' and 'kernel-PAE' are deemed 'performance' kernels, and hence have no debugging options enabled which impact performance. Several low impact options remain enabled, such as
- DEBUG_LIST
- SPINLOCK_SLEEP
- DEBUG_SHIRQ
- DEBUG_RODATA.
In addition, SLUB_DEBUG is enabled, but by default is inactive. You need to boot with slub_debug=1 to make it perform its usual checks.
'kernel-debug' and 'kernel-PAE-debug' also enable CONFIG_SLUB_DEBUG_ON, which means you don't need to boot with slub_debug=, instead it's always on. (It can be disabled with slub_debug=-) The numerous lock dependancy checker options are enabled. Most of the cost of this option is due to the size of spinlock/mutex structures increasing. If embedded into other structures, these can blow up considerably. For performance critical structures like page struct (which normally fits in a cacheline), this can be expensive. DEBUG_VM is also enabled in the debug builds.
Finally, the -debug kernels enable a bunch of fault-injection test modules.
Rawhide
For the most part, the same as 'kernel-debug'. Main differences include
- CONFIG_DEBUG_IGNORE_QUIET is enabled, which makes the 'quiet' boot parameter ineffective. This is done to ease debugging.
- For the first alpha releases, DEBUG_PAGEALLOC is likely to be set, which is incredibly performance taxing. It's also known to cause problems on some virtual machines with buggy pagefault handlers.
- Later alpha/beta releases disable PAGEALLOC in favour of relying on SLUB_DEBUG to catch similar bugs. If obscure hard-to-debug issues occur later in the development cycle, PAGEALLOC may be re-enabled temporarily.