KernelBugTriage

From FedoraProject

(Difference between revisions)
Jump to: navigation, search
(add inital video thing)
(Quick links to bug lists)
(25 intermediate revisions by 7 users not shown)
Line 1: Line 1:
{{admon/important|This page is a draft only|It is still under construction and content may change. Do not rely on the information on this page.}}
 
 
 
== Quick links to bug lists ==
 
== Quick links to bug lists ==
  
* [https://bugzilla.redhat.com/bugzilla/buglist.cgi?product=Fedora&version=11&component=kernel&bug_status=NEW&bug_status=VERIFIED&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=NEEDINFO&bug_status=MODIFIED&bug_status=ASSIGN_TO_PM&bug_status=INVESTIGATE&bug_status=ON_DEV&bug_status=QA_READY&bug_status=ON_QA&bug_status=FAILS_QA&bug_status=UNCONFIRMED&bug_status=NEEDINFO_REPORTER&bug_status=NEEDINFO_PM&bug_status=NEEDINFO_ENG&bug_status=PASSES_QA&bug_status=RELEASE_PENDING&bug_status=POST&bug_status=NEEDINFO_QA&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=    Fedora 11 Kernel Bugs]  
+
* [https://bugzilla.redhat.com/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=MODIFIED&bug_status=ON_DEV&bug_status=ON_QA&bug_status=VERIFIED&bug_status=RELEASE_PENDING&bug_status=POST&classification=Fedora&component=kernel&product=Fedora&query_format=advanced&version=17&order=bug_id%20DESC&query_based_on=    Fedora 17 Kernel Bugs]  
* [https://bugzilla.redhat.com/bugzilla/buglist.cgi?product=Fedora&version=12&component=kernel&bug_status=NEW&bug_status=VERIFIED&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=NEEDINFO&bug_status=MODIFIED&bug_status=ASSIGN_TO_PM&bug_status=INVESTIGATE&bug_status=ON_DEV&bug_status=QA_READY&bug_status=ON_QA&bug_status=FAILS_QA&bug_status=UNCONFIRMED&bug_status=NEEDINFO_REPORTER&bug_status=NEEDINFO_PM&bug_status=NEEDINFO_ENG&bug_status=PASSES_QA&bug_status=RELEASE_PENDING&bug_status=POST&bug_status=NEEDINFO_QA&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=    Fedora 12 Kernel Bugs]  
+
 
* [https://bugzilla.redhat.com/bugzilla/buglist.cgi?product=Fedora&version=13&component=kernel&bug_status=NEW&bug_status=VERIFIED&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=NEEDINFO&bug_status=MODIFIED&bug_status=ASSIGN_TO_PM&bug_status=INVESTIGATE&bug_status=ON_DEV&bug_status=QA_READY&bug_status=ON_QA&bug_status=FAILS_QA&bug_status=UNCONFIRMED&bug_status=NEEDINFO_REPORTER&bug_status=NEEDINFO_PM&bug_status=NEEDINFO_ENG&bug_status=PASSES_QA&bug_status=RELEASE_PENDING&bug_status=POST&bug_status=NEEDINFO_QA&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=    Fedora 13 Kernel Bugs]  
+
* [https://bugzilla.redhat.com/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=MODIFIED&bug_status=ON_DEV&bug_status=ON_QA&bug_status=VERIFIED&bug_status=RELEASE_PENDING&bug_status=POST&classification=Fedora&component=kernel&product=Fedora&query_format=advanced&version=18&order=bug_id%20DESC&query_based_on=    Fedora 18 Kernel Bugs]  
* [https://bugzilla.redhat.com/bugzilla/buglist.cgi?product=Fedora&version=rawhide&component=kernel&bug_status=NEW&bug_status=VERIFIED&bug_status=ASSIGNED&bug_status=REOPENED&bug_status=NEEDINFO&bug_status=MODIFIED&bug_status=ASSIGN_TO_PM&bug_status=INVESTIGATE&bug_status=ON_DEV&bug_status=QA_READY&bug_status=ON_QA&bug_status=FAILS_QA&bug_status=UNCONFIRMED&bug_status=NEEDINFO_REPORTER&bug_status=NEEDINFO_PM&bug_status=NEEDINFO_ENG&bug_status=PASSES_QA&bug_status=RELEASE_PENDING&bug_status=POST&bug_status=NEEDINFO_QA&short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=    Fedora Rawhide Kernel Bugs]  
+
 
 +
* [https://bugzilla.redhat.com/buglist.cgi?bug_status=NEW&bug_status=ASSIGNED&bug_status=MODIFIED&bug_status=ON_DEV&bug_status=ON_QA&bug_status=VERIFIED&bug_status=RELEASE_PENDING&bug_status=POST&classification=Fedora&component=kernel&product=Fedora&query_format=advanced&version=rawhide&order=bug_id%20DESC&query_based_on=    Fedora rawhide Kernel Bugs]
 +
 
 +
* [https://bugzilla.redhat.com/buglist.cgi?keywords=Security&keywords_type=anywords&order=Bug%20Number&classification=Fedora&query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&bug_status=MODIFIED&bug_status=ON_DEV&bug_status=ON_QA&bug_status=VERIFIED&bug_status=RELEASE_PENDING&bug_status=POST&version=14&version=15&version=16&version=rawhide&component=kernel&product=Fedora     Fedora Kernel Security Bugs (all versions)]
 +
 
 
* [http://bugzilla.kernel.org/buglist.cgi?short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=&kernel_version_type=allwordssubstr&kernel_version=&bug_status=NEW&bug_status=ASSIGNED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&changedin=&chfieldfrom=&chfieldto=Now&chfieldvalue=&newqueryname=&field0-0-0=noop&type0-0-0=noop&value0-0-0=  Upstream Kernel Bugs]
 
* [http://bugzilla.kernel.org/buglist.cgi?short_desc_type=allwordssubstr&short_desc=&long_desc_type=allwordssubstr&long_desc=&kernel_version_type=allwordssubstr&kernel_version=&bug_status=NEW&bug_status=ASSIGNED&emailassigned_to1=1&emailtype1=substring&email1=&emailassigned_to2=1&emailreporter2=1&emailcc2=1&emailtype2=substring&email2=&bugidtype=include&bug_id=&changedin=&chfieldfrom=&chfieldto=Now&chfieldvalue=&newqueryname=&field0-0-0=noop&type0-0-0=noop&value0-0-0=  Upstream Kernel Bugs]
  
Line 20: Line 22:
 
:''Thank you for taking the time to report this bug. Bugs involving the installation process cannot usually be resolved after release date. We therefore ask reporters to use one (or both) of the following options:
 
:''Thank you for taking the time to report this bug. Bugs involving the installation process cannot usually be resolved after release date. We therefore ask reporters to use one (or both) of the following options:
  
:''1. Test with a Fedora Unity re-spin. These can be found at http://spins.fedoraunity.org/spins and include updates for installer bugs which may resolve the problem for you.''
+
:''Test with an Alpha, Beta or Release Candidate for the next version. These can be found at http://fedoraproject.org/get-prerelease and by testing these we can work to resolve any installation issues so that the final release is free of such bugs.''
:''2. Test with an Alpha, Beta or Release Candidate for the next version. These can be found at http://fedoraproject.org/get-prerelease and by testing these we can work to resolve any installation issues so that the final release is free of such bugs.''
+
  
 
:''However if the problem no longer exists then please close this bug or it will be closed in a few weeks if there is no additional information lodged.''
 
:''However if the problem no longer exists then please close this bug or it will be closed in a few weeks if there is no additional information lodged.''
  
{{admon/important|Outdated fedoraunity spins|The spins at fedoraunity are not updated since 2010-03-03.}}
 
 
{{admon/important|get-prerelease link|The ''get-prerelease'' link sometimes is not available or have not updated development releases.}}
 
{{admon/important|get-prerelease link|The ''get-prerelease'' link sometimes is not available or have not updated development releases.}}
  
Line 56: Line 56:
 
* X bugs see: http://fedoraproject.org/wiki/How_to_debug_Xorg_problems.
 
* X bugs see: http://fedoraproject.org/wiki/How_to_debug_Xorg_problems.
 
* Other troubleshooting ideas: http://fedoraproject.org/wiki/Common_kernel_problems.
 
* Other troubleshooting ideas: http://fedoraproject.org/wiki/Common_kernel_problems.
 +
 +
== Rebase Regressions ==
 +
The kernel is often rebased to the latest stable kernel release and pushed as an update.  This fixes a large amount of bugs, but it inevitably introduces a few regressions.  If a bug is reported and it is determined that it worked with the previous kernel release, then a rebase regression entry should be added to the whiteboard field in the bug. This entry takes the form of:
 +
 +
<pre>rebase-regression-<kernel version></pre>
 +
 +
E.g. if there is a regression when moving to the 3.1 kernel, it should be:
 +
 +
<pre>rebase-regression-3.1</pre>
 +
 +
The latest working Fedora kernel version and the earliest broken version should be noted in the comment section if possible.
 +
 +
This will allow the kernel developers to track regressions introduced by a rebased kernel.  That will also help facilitate our communication with the upstream kernel developers, as we can report these bugs to them to be included in the regression lists that are published upstream.
  
 
== Bugs with TAINTED modules ==
 
== Bugs with TAINTED modules ==
Line 76: Line 89:
  
 
* If there's an upstream (http://bugzilla.kernel.org) bug that matches
 
* If there's an upstream (http://bugzilla.kernel.org) bug that matches
* In the Fedora bug, set 'external bugzilla references' to point to the upstream bug number
+
** In the Fedora bug, set 'external bugzilla references' to point to the upstream bug number
* In the upstream kernel.org bugzilla, add a comment along the lines of  "also seen in Fedora. http://bugzilla.redhat.com/show_bug.cgi?id=253096"
+
** In the upstream kernel.org bugzilla, add a comment along the lines of  "also seen in Fedora. http://bugzilla.redhat.com/show_bug.cgi?id=253096"
* If the report contains a lockdep trace, mark it as blocking FCMETA_LOCKDEP
+
 
* If the bug is against an already released Fedora (Ie, Fedora 7) and there isn't much information to go on "my machine locked up" for eg, request that the user try to reproduce the problem using the kernel-debug kernel. The additional debugging checks may trigger some clues.
 
* If the bug is against an already released Fedora (Ie, Fedora 7) and there isn't much information to go on "my machine locked up" for eg, request that the user try to reproduce the problem using the kernel-debug kernel. The additional debugging checks may trigger some clues.
 
* If the bug you are triaging contains a patch, please add [PATCH]  to the beginning of the summary line of the bugzilla.
 
* If the bug you are triaging contains a patch, please add [PATCH]  to the beginning of the summary line of the bugzilla.
 
* Sometimes a user will attach a kernel panic as a jpeg, or in worst case, as a tarball of their /var/log/messages.
 
* Sometimes a user will attach a kernel panic as a jpeg, or in worst case, as a tarball of their /var/log/messages.
* It saves the kernel team time if the kernel oops parts of these are transcribed/cut out and pasted into the bug as text.
+
** It saves the kernel team time if the kernel oops parts of these are transcribed/cut out and pasted into the bug as text.
 
* Additionally, making sure that text attachments of bugs have their MIME type set to text/plain can save some time.
 
* Additionally, making sure that text attachments of bugs have their MIME type set to text/plain can save some time.
 
* You can ask reporters to try and duplicate with the latest nightly live composes: http://alt.fedoraproject.org/pub/alt/nightly-composes/
 
* You can ask reporters to try and duplicate with the latest nightly live composes: http://alt.fedoraproject.org/pub/alt/nightly-composes/
Line 90: Line 102:
  
 
{{Anchor|common_problems}}
 
{{Anchor|common_problems}}
 +
 
== Common Problems ==
 
== Common Problems ==
 
Point reporters to: [[KernelCommonProblems]]
 
Point reporters to: [[KernelCommonProblems]]
  
 
== Bug assignment ==
 
== Bug assignment ==
For certain categories of bug, assigning/cc'ing people responsible for the relevant subsystem is a good idea.
+
Assigning/cc'ing people responsible for the relevant subsystem is a good idea.
When reassigning Add kernel-maint@redhat.com to Cc:
+
 
 +
When reassigning bugs to yourself, add kernel-maint@redhat.com to Cc:
  
{| border="1"
+
If you are an upstream developer, and would like to be added to the Cc: of relevant bugs, send an email to kernel-team at fedoraproject.org,
|-
+
(We used to maintain this here on the wiki, but for a number of reasons, we now maintain this list privately)
| Subsystem || Reassign to || Add to Cc: || Other notes.
+
|-
+
| intel video || xorg-x11-drv-intel || ||
+
|-
+
| radeon video || xorg-x11-drv-radeon || ||
+
|-
+
| nouveau video || xorg-x11-drv-ati || ||
+
|-
+
| Installer || || bob@fedoraunity.org, kanarip@kanarip.com, jbwillia@math.vt.edu ||
+
|-
+
| Dell-specific || || linux-bugs@dell.com ||
+
|-
+
| ACPI || || acpi-bugzilla@lists.sourceforge.net || Add FCMETA_ACPI to 'blocks'
+
|-
+
| ELF interpreter || roland@redhat.com || ||
+
|-
+
| PATA ||  || ||
+
|-
+
| SATA || jgarzik@redhat.com || || * Add FCMETA_SATA to 'blocks'
+
|-
+
| USB || || zaitcev@redhat.com || Add FCMETA_USB to 'blocks'
+
|-
+
| SCSI || || || Add FCMETA_SCSI to 'blocks'
+
|-
+
| device-mapper || lvm-team@redhat.com || ||
+
|-
+
| Sound || || || Add FCMETA_ALSA to 'blocks'
+
|-
+
| Firewire || jwilson@redhat.com || stefan-r-rhbz@s5r6.in-berlin.de ||
+
|-
+
| Wireless || linville@redhat.com || ||
+
|-
+
| Broadcom wireless driver (bcm43xx) || linville@redhat.com || larry.finger@lwfinger.net ||
+
|-
+
| Broadcom wired drivers (bnx2, bnx2x, b44, tg3) || || mcarlson@broadcom.com ||
+
|-
+
| CIFS || jlayton@redhat.com || steved@redhat.com, staubach@redhat.com ||
+
|-
+
| EXT2/EXT3/JBD/XFS || esandeen@redhat.com || || (Or pretty much any filesystem stuff goes to Eric)
+
|-
+
| CPU frequency scaling || davej@redhat.com || anton@redhat.com ||
+
|-
+
| 3D (DRM/AGP) || airlied@redhat.com || || Change component to xorg-x11-drv-{intel,ati}
+
|-
+
| udev related || || kay.sievers@vrfy.org ||
+
|-
+
| USB cameras || hdegoede@redhat.com || ||
+
|-
+
| KVM/Xen || fedora-virt-maint@redhat.com ||  ||
+
|-
+
| ptrace (and utrace) || roland@redhat.com ||  ||
+
|-
+
| inotify || eparis@redhat.com ||  ||
+
|-
+
| IMA (integrity measurement) || eparis@redhat.com ||  ||
+
|-
+
| DMAR (Intel IOMMU) || dwmw2@infradead.org ||  ||
+
  
 
[[Category:Debugging]]
 
[[Category:Debugging]]

Revision as of 16:09, 7 March 2013

Contents

Quick links to bug lists

Possibly fixed in CURRENTRELEASE (Stick for prodding sleeping bugs with)

If the bug is against a previous point release kernel, and the reporter hasn't re-tested (this type of bugs must be set in NEEDINFO_REPORTER state):

Thank you for taking the time to report this bug. Updates to the kernel have been released since it was first reported. If you have time to update the package and re-test, please do so and report the results here. You can obtain the updated package by typing 'yum update kernel' or using the graphical updater, Software Update. If the problem no longer exists then please close this bug or it will be closed in a few weeks if there is no additional information lodged.

Installer bugs

If the bug refers to problems when installing Fedora in any of the possible ways CD, DVD, USB, network, etc (this type of bugs must be set in NEEDINFO_REPORTER state):

Thank you for taking the time to report this bug. Bugs involving the installation process cannot usually be resolved after release date. We therefore ask reporters to use one (or both) of the following options:
Test with an Alpha, Beta or Release Candidate for the next version. These can be found at http://fedoraproject.org/get-prerelease and by testing these we can work to resolve any installation issues so that the final release is free of such bugs.
However if the problem no longer exists then please close this bug or it will be closed in a few weeks if there is no additional information lodged.
Important.png
get-prerelease link
The get-prerelease link sometimes is not available or have not updated development releases.

Video Subsystem bugs

The various video subsystems that have kernel modesetting support would just prefer their bugs to be moved over to the xorg-x11-drv-{intel|nouveau|ati} component. There, their triagers can gather needed info and debug the issue. If the problem is in the kernel, those maintainers can fix it, no need to leave the bug assigned to the kernel.

This bug is in a video subsystem that has a kernel part. We track and work on these bugs via the driver package name instead of leaving them assigned to the kernel

NEW state bugs

If the bug is new and (this type of bugs must be key-worded in either WHITEBOARD or BLOCKER): Hello,

Thank you for taking the time to report this bug. I have added the proper keyword to this bug to bring it to the attention of the <subsystem-name> subsystem maintainer.
Can you attach the following information to this bug:

Then pick the info that is useful in this case:

  • smolt profile
  • dmesg output from boot before issue.
  • dmesg output after issue.
  • dmesg output from boot with 'quiet' option replaced with 'debug'
  • lspci output
  • lsusb output

For other necessary information check:

Rebase Regressions

The kernel is often rebased to the latest stable kernel release and pushed as an update. This fixes a large amount of bugs, but it inevitably introduces a few regressions. If a bug is reported and it is determined that it worked with the previous kernel release, then a rebase regression entry should be added to the whiteboard field in the bug. This entry takes the form of:

rebase-regression-<kernel version>

E.g. if there is a regression when moving to the 3.1 kernel, it should be:

rebase-regression-3.1

The latest working Fedora kernel version and the earliest broken version should be noted in the comment section if possible.

This will allow the kernel developers to track regressions introduced by a rebased kernel. That will also help facilitate our communication with the upstream kernel developers, as we can report these bugs to them to be included in the regression lists that are published upstream.

Bugs with TAINTED modules

Certain bug reports refer to a kernel module that is not open source software (mark this bugs with the keyword TAINTED):

Thank you for taking the time to report this bug. Sadly, your report contains information about a not open-sourced module. This means you have loaded a module into your kernel that we can't debug, making it difficult to isolate the issue. Can you reproduce the issue without loading the tainted <module-name> module?.
If you cannot duplicate the bug without the tainted module loaded it will be closed.
However, you can report it to the vendor of your non-free module.

Bug state transitions

  • A bug marked as MODIFIED has patches in testing and should not have their status changed. An exception to this is when a bug has been in MODIFIED state for some time (this usually indicates that the issue was fixed in an update, and no-one ever closed the bug. If in doubt, ping the reporter to retest with the latest build).
  • If a bug has been in NEEDINFO for several months with no follow-up from the reporter, there's not a great deal we can do. In this situation, closing the bug is the only recourse available (which does actually tend to 'wake up' some reporters who then reopen the bug).
  • When marking bugs as duplicates, we want the one with the most information to be the one that remains open, even if this means a higher numbered bug gets closed.
  • Don't close bugs marked beginning with [mmconf] or [msi] (bugs beginning with [$something] are usually specifically marked so developers can quickly see the main issue within the bug)

Basics

  • If there's an upstream (http://bugzilla.kernel.org) bug that matches
  • If the bug is against an already released Fedora (Ie, Fedora 7) and there isn't much information to go on "my machine locked up" for eg, request that the user try to reproduce the problem using the kernel-debug kernel. The additional debugging checks may trigger some clues.
  • If the bug you are triaging contains a patch, please add [PATCH] to the beginning of the summary line of the bugzilla.
  • Sometimes a user will attach a kernel panic as a jpeg, or in worst case, as a tarball of their /var/log/messages.
    • It saves the kernel team time if the kernel oops parts of these are transcribed/cut out and pasted into the bug as text.
  • Additionally, making sure that text attachments of bugs have their MIME type set to text/plain can save some time.
  • You can ask reporters to try and duplicate with the latest nightly live composes: http://alt.fedoraproject.org/pub/alt/nightly-composes/
  • You can ask reporters to try the newest kernel available in koji: http://koji.fedoraproject.org/koji/packageinfo?packageID=8
  • If the bug appears to be video/kernel modesetting related, reassign it to the xorg-x11-drv-{intel,ati,nouveau} component.
  • If the reporter is using a custom compiled kernel, close the bug as a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=126342

Common Problems

Point reporters to: KernelCommonProblems

Bug assignment

Assigning/cc'ing people responsible for the relevant subsystem is a good idea.

When reassigning bugs to yourself, add kernel-maint@redhat.com to Cc:

If you are an upstream developer, and would like to be added to the Cc: of relevant bugs, send an email to kernel-team at fedoraproject.org, (We used to maintain this here on the wiki, but for a number of reasons, we now maintain this list privately)