(create fwn 217 qa beat) |
(create fwn 218 qa beat) |
||
| Line 2: | Line 2: | ||
== QualityAssurance == | == QualityAssurance == | ||
In this section, we cover the activities of the QA team<ref>http://fedoraproject.org/wiki/QA</ref>. | In this section, we cover the activities of the QA team<ref>http://fedoraproject.org/wiki/QA</ref>. This week, we are trying out a new topic-focused layout, without the topic-by-topic weekly meeting recaps. Please let [[User:Adamwill|me]] know if you particularly like or dislike the new layout! | ||
Contributing Writer: [[User:Adamwill|Adam Williamson]] | Contributing Writer: [[User:Adamwill|Adam Williamson]] | ||
| Line 10: | Line 10: | ||
=== Test Days === | === Test Days === | ||
Last week's Test Day<ref> | Last week's Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2010-03-18_Palimpsest</ref> was on Fedora 13 changes to disk management, via the udisks (previously DeviceKit-disks) backend and the Palimpsest front end<ref>http://fedoraproject.org/wiki/Features/UdisksImprovements</ref>. The turnout was solid and resulted in 12 bugs being filed, of which two have already been fixed. [[User:Davidz|David Zeuthen]] was there to help with testing throughout the test day and will be working to fix the remaining bugs. | ||
Next week's Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2010-03- | Next week's Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2010-03-25_Printing</ref> will be on printing, including the implementation of automatic print driver installation<ref>http://fedoraproject.org/wiki/Features/AutomaticPrintDriverInstallation</ref>in Fedora 13. This is a nice mix of making sure the existing printer support is working well and testing out a shiny new feature. Printing is important to nearly everyone, so if you've got a printer, please come along and help test! As usual, you can test with an installed Fedora 13 or Rawhide system, or a live image which is available on the Test Day page. [[User:twaugh|Tim Waugh]] has been working hard to arrange the test day, and he and Jiri Popelka, along with QA's [[User:Ykopkova|Yulia Kopkova]], will be on hand to help with any questions or problems. The Test Day will run all day on Thursday 2010-03-25 in the #fedora-test-day IRC channel. | ||
If you would like to propose a main track Test Day for the Fedora 13 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>. | If you would like to propose a main track Test Day for the Fedora 13 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>. | ||
| Line 18: | Line 18: | ||
<references/> | <references/> | ||
=== | === Fedora 13 testing === | ||
The first acceptance test plan<ref>http://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan</ref> run of the Beta period was performed on the Fedora 13 tree of 2010-03-15<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_13_Pre-Beta_Acceptance_Test_1</ref>, resulting in a pass, with two incidental issues noted and reported. | |||
During the weekly QA meeting<ref>http://fedoraproject.org/wiki/QA/Meetings/20100315</ref>, [[User:Jlaska|James Laska]] noted that release engineering should be working on the first Beta test compose for 2010-03-18, but it did not arrive during the week. [[User:Rhe|Rui He]] announced<ref>http://lists.fedoraproject.org/pipermail/test/2010-March/089340.html</ref> the installation and desktop acceptance testing matrices for the candidate in preparation for its expected arrival. | |||
The second blocker bug review meeting for Fedora 13 Beta was held on 2010-03-19<ref>http://meetbot.fedoraproject.org/fedora-bugzappers/2010-03-19/f13beta-blocker-review.2010-03-19-16.04.html</ref>. All outstanding Beta blocker bugs were reviewed, and developers were consulted on the remaining open bugs to ensure fixes should be available in time for the release candidate process to begin the following week. | |||
Robert P.J. Day<ref>http://lists.fedoraproject.org/pipermail/test/2010-March/089175.html</ref> and Felix Miata<ref>http://lists.fedoraproject.org/pipermail/test/2010-March/089297.html</ref> noted issues with the package customization choices in the Fedora 13 installation process. Robert noticed that the package group customization screen only allowed one major package group to be chosen, leading to a discussion on whether this was a sensible decision or whether it should be possible to select multiple high-level groups. [[User:dcantrel|David Cantrell]] explained<ref>http://lists.fedoraproject.org/pipermail/test/2010-March/089213.html</ref> that the Anaconda team had noticed that, in previous releases where multiple selections were allowed, "most users would either (a) leave the default | |||
choices in place and continue or (b) check them all for fear of missing something". However, he was still open to the possibility of changing to multiple selection after "look[ing] at the tasks as | |||
defined now [to] see if they should be changed, expanded, or reduced". | |||
[[User: | Felix noticed that the minimal installation option resulted in a non-working network connection on first boot. [[User:jkeating|Jesse Keating]] explained<ref>http://lists.fedoraproject.org/pipermail/test/2010-March/089307.html</ref> that this is because the minimal install does not include NetworkManager, and the installer does not have code to detect when this is the case and enable the traditional-style ''network'' service instead. Some debate ensued over whether this feature should be implemented, and about whether minimal installations had had working network connections or not on earlier Fedora releases. | ||
Several group members, including Jim Haynes<ref>http://lists.fedoraproject.org/pipermail/test/2010-March/089210.html</ref>, Tom Horsley<ref>http://lists.fedoraproject.org/pipermail/test/2010-March/089223.html</ref> and Robert Lightfoot<ref>http://lists.fedoraproject.org/pipermail/test/2010-March/089385.html</ref>, were engaged in testing Fedora 13 images and gave some valuable feedback on the issues they encountered. | |||
<references/> | <references/> | ||
=== | === Update acceptance testing === | ||
[[User: | During the weekly QA meeting, [[User:Jlaska|James Laska]] noted that the group needed to make progress on defining a process for update acceptance testing. Packages intended for Fedora 13 were already being held in the updates-testing repository for testing, and FESCo had agreed to a policy requiring proposed updates for critical path packages in stable releases to be held for testing as well. The QA group had the responsibility to decide how the group of trusted testers whose feedback would be used to approve updates should be defined. [[User:maxamillion|Adam Miller]] had already provided an initial draft policy<ref>http://lists.fedoraproject.org/pipermail/test/2010-March/088980.html</ref>, but this was waiting on feedback from the rest of the group. | ||
<references/> | <references/> | ||
=== | === Privilege escalation testing === | ||
[[User: | During the weekly QA meeting, [[User:Adamwill|Adam Williamson]] noted that, since the privilege escalation policy<ref>http://fedoraproject.org/wiki/Privilege_escalation_policy</ref> which the group had worked on had been accepted, no progress had yet been made on implementing testing. He felt that the logical next step would be to produce the planned script to identify packages which potentially perform privilege escalation; this would give an idea of the problem space and potentially allow basic manual testing during the Fedora 13 cycle. He planned to talk to [[User:Wwoods|Will Woods]] to see if this could be implemented quickly. | ||
<references/> | |||
=== New release process Wiki documentation === | |||
During the weekly Bugzappers meeting<ref>http://meetbot.fedoraproject.org/fedora-meeting/2010-03-16/fedora-meeting.2010-03-16-15.03.log.html</ref>, [[User:Beland|Christopher Beland]] announced that he had revised several pages on the Wiki to reflect the new, No Frozen Rawhide-based<ref>http://fedoraproject.org/wiki/No_Frozen_Rawhide_Proposal</ref> release process. He had updated the Rawhide<ref>http://fedoraproject.org/wiki/Releases/Rawhide</ref> and Fedora Release Life Cycle<ref>http://fedoraproject.org/wiki/Fedora_Release_Life_Cycle</ref> pages, and created a new Branched page<ref>http://fedoraproject.org/wiki/Releases/Branched</ref> to document the ''Branched'' release (which will always be the upcoming stable Fedora release, so is currently Fedora 13). [[User:Jlaska|James Laska]] noted that he had an older proposal<ref>http://fedoraproject.org/wiki/User:Jlaska/Draft</ref> to split the Rawhide page into several smaller pages. James, Christopher and [[User:Adamwill|Adam Williamson]] discussed whether this would be a good idea, and eventually agreed that the Rawhide and Branched pages could benefit from streamlining and possibly from splitting, but detailed proposals should be sent to the mailing list. | |||
<references/> | |||
=== Target bug trackers === | |||
During the weekly Bugzappers meeting, [[User:Beland|Christopher Beland]] raised the question of what should be done with the Target bug trackers - F12Target, F13Target and so on. These are intended to track bugs which are not quite release blockers but are important and should receive greater developer attention in a best effort to resolve them before release. However, the group felt that they are not serving this purpose, and are mostly ignored by developers. They also noted that there was significant overlap with the ''severity'' field, since the group had defined a policy for the use of that field and begun to implement it in triaging. As no-one had ideas for reviving the usefulness of the Target trackers, the group agreed that [[User:Adamwill|Adam Williamson]] should draft a proposal to drop them, to be forward to the development group. | |||
Later in the week, at the blocker bug review meeting, [[User:jkeating|Jesse Keating]] suggested the Target tracker could be used during tight freeze periods to track bugs which were not strictly release blockers, but which QA and release engineering would accept fixes for, through the freeze. | |||
<references/> | <references/> | ||
Revision as of 20:11, 22 March 2010
QualityAssurance
In this section, we cover the activities of the QA team[1]. This week, we are trying out a new topic-focused layout, without the topic-by-topic weekly meeting recaps. Please let me know if you particularly like or dislike the new layout!
Contributing Writer: Adam Williamson
Test Days
Last week's Test Day[1] was on Fedora 13 changes to disk management, via the udisks (previously DeviceKit-disks) backend and the Palimpsest front end[2]. The turnout was solid and resulted in 12 bugs being filed, of which two have already been fixed. David Zeuthen was there to help with testing throughout the test day and will be working to fix the remaining bugs.
Next week's Test Day[3] will be on printing, including the implementation of automatic print driver installation[4]in Fedora 13. This is a nice mix of making sure the existing printer support is working well and testing out a shiny new feature. Printing is important to nearly everyone, so if you've got a printer, please come along and help test! As usual, you can test with an installed Fedora 13 or Rawhide system, or a live image which is available on the Test Day page. Tim Waugh has been working hard to arrange the test day, and he and Jiri Popelka, along with QA's Yulia Kopkova, will be on hand to help with any questions or problems. The Test Day will run all day on Thursday 2010-03-25 in the #fedora-test-day IRC channel.
If you would like to propose a main track Test Day for the Fedora 13 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac[5].
Fedora 13 testing
The first acceptance test plan[1] run of the Beta period was performed on the Fedora 13 tree of 2010-03-15[2], resulting in a pass, with two incidental issues noted and reported.
During the weekly QA meeting[3], James Laska noted that release engineering should be working on the first Beta test compose for 2010-03-18, but it did not arrive during the week. Rui He announced[4] the installation and desktop acceptance testing matrices for the candidate in preparation for its expected arrival.
The second blocker bug review meeting for Fedora 13 Beta was held on 2010-03-19[5]. All outstanding Beta blocker bugs were reviewed, and developers were consulted on the remaining open bugs to ensure fixes should be available in time for the release candidate process to begin the following week.
Robert P.J. Day[6] and Felix Miata[7] noted issues with the package customization choices in the Fedora 13 installation process. Robert noticed that the package group customization screen only allowed one major package group to be chosen, leading to a discussion on whether this was a sensible decision or whether it should be possible to select multiple high-level groups. David Cantrell explained[8] that the Anaconda team had noticed that, in previous releases where multiple selections were allowed, "most users would either (a) leave the default choices in place and continue or (b) check them all for fear of missing something". However, he was still open to the possibility of changing to multiple selection after "look[ing] at the tasks as defined now [to] see if they should be changed, expanded, or reduced".
Felix noticed that the minimal installation option resulted in a non-working network connection on first boot. Jesse Keating explained[9] that this is because the minimal install does not include NetworkManager, and the installer does not have code to detect when this is the case and enable the traditional-style network service instead. Some debate ensued over whether this feature should be implemented, and about whether minimal installations had had working network connections or not on earlier Fedora releases.
Several group members, including Jim Haynes[10], Tom Horsley[11] and Robert Lightfoot[12], were engaged in testing Fedora 13 images and gave some valuable feedback on the issues they encountered.
- ↑ http://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan
- ↑ http://fedoraproject.org/wiki/Test_Results:Fedora_13_Pre-Beta_Acceptance_Test_1
- ↑ http://fedoraproject.org/wiki/QA/Meetings/20100315
- ↑ http://lists.fedoraproject.org/pipermail/test/2010-March/089340.html
- ↑ http://meetbot.fedoraproject.org/fedora-bugzappers/2010-03-19/f13beta-blocker-review.2010-03-19-16.04.html
- ↑ http://lists.fedoraproject.org/pipermail/test/2010-March/089175.html
- ↑ http://lists.fedoraproject.org/pipermail/test/2010-March/089297.html
- ↑ http://lists.fedoraproject.org/pipermail/test/2010-March/089213.html
- ↑ http://lists.fedoraproject.org/pipermail/test/2010-March/089307.html
- ↑ http://lists.fedoraproject.org/pipermail/test/2010-March/089210.html
- ↑ http://lists.fedoraproject.org/pipermail/test/2010-March/089223.html
- ↑ http://lists.fedoraproject.org/pipermail/test/2010-March/089385.html
Update acceptance testing
During the weekly QA meeting, James Laska noted that the group needed to make progress on defining a process for update acceptance testing. Packages intended for Fedora 13 were already being held in the updates-testing repository for testing, and FESCo had agreed to a policy requiring proposed updates for critical path packages in stable releases to be held for testing as well. The QA group had the responsibility to decide how the group of trusted testers whose feedback would be used to approve updates should be defined. Adam Miller had already provided an initial draft policy[1], but this was waiting on feedback from the rest of the group.
Privilege escalation testing
During the weekly QA meeting, Adam Williamson noted that, since the privilege escalation policy[1] which the group had worked on had been accepted, no progress had yet been made on implementing testing. He felt that the logical next step would be to produce the planned script to identify packages which potentially perform privilege escalation; this would give an idea of the problem space and potentially allow basic manual testing during the Fedora 13 cycle. He planned to talk to Will Woods to see if this could be implemented quickly.
New release process Wiki documentation
During the weekly Bugzappers meeting[1], Christopher Beland announced that he had revised several pages on the Wiki to reflect the new, No Frozen Rawhide-based[2] release process. He had updated the Rawhide[3] and Fedora Release Life Cycle[4] pages, and created a new Branched page[5] to document the Branched release (which will always be the upcoming stable Fedora release, so is currently Fedora 13). James Laska noted that he had an older proposal[6] to split the Rawhide page into several smaller pages. James, Christopher and Adam Williamson discussed whether this would be a good idea, and eventually agreed that the Rawhide and Branched pages could benefit from streamlining and possibly from splitting, but detailed proposals should be sent to the mailing list.
- ↑ http://meetbot.fedoraproject.org/fedora-meeting/2010-03-16/fedora-meeting.2010-03-16-15.03.log.html
- ↑ http://fedoraproject.org/wiki/No_Frozen_Rawhide_Proposal
- ↑ http://fedoraproject.org/wiki/Releases/Rawhide
- ↑ http://fedoraproject.org/wiki/Fedora_Release_Life_Cycle
- ↑ http://fedoraproject.org/wiki/Releases/Branched
- ↑ http://fedoraproject.org/wiki/User:Jlaska/Draft
Target bug trackers
During the weekly Bugzappers meeting, Christopher Beland raised the question of what should be done with the Target bug trackers - F12Target, F13Target and so on. These are intended to track bugs which are not quite release blockers but are important and should receive greater developer attention in a best effort to resolve them before release. However, the group felt that they are not serving this purpose, and are mostly ignored by developers. They also noted that there was significant overlap with the severity field, since the group had defined a policy for the use of that field and begun to implement it in triaging. As no-one had ideas for reviving the usefulness of the Target trackers, the group agreed that Adam Williamson should draft a proposal to drop them, to be forward to the development group.
Later in the week, at the blocker bug review meeting, Jesse Keating suggested the Target tracker could be used during tight freeze periods to track bugs which were not strictly release blockers, but which QA and release engineering would accept fixes for, through the freeze.