(fwn 214 beat) |
(create fwn 215 qa beat) |
||
| Line 10: | Line 10: | ||
=== Test Days === | === Test Days === | ||
Last week's Test Day was on | Last week's Test Day<ref>http://fedoraproject.org/wiki/Test_Day:2010-02-25_YumLangpackPlugin</ref> was on the langpack plugin for yum<ref>http://fedoraproject.org/wiki/Features/YumLangpackPlugin</ref>. The event suffered a little from falling in the same week as the Alpha candidate testing process, but testing did result in the identification of two bugs. Thanks to those who came out to test. | ||
Next week's Test Day<ref> | Next week's Test Day<ref>https://fedoraproject.org/wiki/Test_Day:2010-03-04_SSSDByDefault</ref> will be on the use of SSSD by default<ref>https://fedoraproject.org/wiki/Features/SSSDByDefault</ref>. As the page says, "The prime benefit of the System Security Services Daemon is support for offline logins. Above and beyond the traditional pam_ldap or pam_krb5 approaches, the SSSD would remove the need for laptop users of Fedora to maintain a local account, separate from their centrally-managed account, to work offline or disconnected from the central servers." This is a significant feature for many people, so please come out and help us test it! The Test Day will run all day on Thursday 2010-03-04 in the #fedora-test-day IRC channel. | ||
If you would like to propose a main track Test Day for the Fedora 13 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>. | If you would like to propose a main track Test Day for the Fedora 13 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac<ref>http://fedorahosted.org/fedora-qa/</ref>. | ||
| Line 20: | Line 20: | ||
=== Weekly meetings === | === Weekly meetings === | ||
The QA group weekly meeting<ref>http://fedoraproject.org/wiki/QA/Meetings</ref> was held on 2010-02- | The QA group weekly meeting<ref>http://fedoraproject.org/wiki/QA/Meetings</ref> was held on 2010-02-22. The full logs are available<ref>http://fedoraproject.org/wiki/QA/Meetings/20100222</ref>. [[User:Adamwill|Adam Williamson]] noted that the privilege escalation policy<ref>http://fedoraproject.org/wiki/Privilege_escalation_policy</ref> the group had helped to create had been approved by FESco and was now an official policy. | ||
[[User:Jlaska|James Laska]] | [[User:Jlaska|James Laska]] noted that the first Alpha release candidate was now available for testing<ref>http://lists.fedoraproject.org/pipermail/test-announce/2010-February/000023.html</ref>. [[User:jkeating|Jesse Keating]] mentioned that it had turned out to be quite broken. James pointed out the test result matrix<ref>http://fedoraproject.org/wiki/Test_Results:Fedora_13_Alpha_RC1_Install#Test_Matrix</ref> as the best summary of exactly how broken. The group briefly discussed the causes for the regressions from the second test compose, with the X server input switch from HAL to dbus being one of the major causes, and some errors in anaconda changes the other. [[User:Kparal|Kamil Paral]] pointed out that an SELinux bug was impeding desktop validation testing. After some discussion, the group agreed testing could go ahead with SELinux disabled and negative results recorded, but not positive results (as it was impossible to be sure if the result would also be positive with SELinux enabled). The group also discussed the best approach to testing to try and get all critical issues fixed without having to delay the Alpha release, and agreed to focus on closing as many open bugs as possible even with the known-bad RC1 images and then testing RC2 quickly once it was made available. | ||
[[User: | [[User:Adamwill|Adam Williamson]] briefly mentioned that he would be putting the topicof Fedora 13 bug procedures up for discussion at the next day's Bugzappers meeting. | ||
The group discussed the question of membership of the QA group in FAS for a while. This has historically been unimportant as it was not used for anything, but in future, positive feedback in Bodhi for critical path packages will be required from members of QA or Release Engineering before the update can be pushed, so the question of group membership becomes important. [[User:maxamillion|Adam Miller]] volunteered to write an initial draft of a policy / procedure on group membership. | |||
[[User: | [[User:maxamillion|Adam Miller]] recapped his call for help with QA brainstorming for the new security spin<ref>http://fedoraproject.org/wiki/SecuritySpin:QA_Brainstorm</ref>. [[User:Jlaska|James Laska]] pointed out that there had recently been a test day for sectool, and suggested those involved with that may be interested in helping. | ||
The Bugzappers group weekly meeting<ref>http://fedoraproject.org/wiki/BugZappers/Meetings</ref> was held on 2010-02-23. The full log is available<ref>http://meetbot.fedoraproject.org/fedora-meeting/2010-02-23/fedora-meeting.2010-02-23-15.04.log.html</ref>. The group discussed Jon Stanley's suggestions<ref>http://lists.fedoraproject.org/pipermail/test/2010-February/088644.html</ref>. No-one could immediately think of any extra resources the project in general could be providing that would help the Bugzappers' work. The question of sending out updates regarding the project's work became a discussion of the long-pending triage statistics project. [[User:Adamwill|Adam Williamson]] said that Brennan Ashton had been discussing incorporating the project into the Fedora Community system at FUDCon. [[User:Tk009|Edward Kirk]] said he had blogged in the last couple of weeks about starting work on this. After the meeting, Brennan contacted the group to let them know he was working actively on the Fedora Community system and had been posting about his work on the infrastructure SIG mailing list. He also granted [[User:Mcepl|Matej Cepl]] administrator access to the triage section on fedorahosted. | |||
[[User: | [[User:Adamwill|Adam Williamson]] introduced the topic of what to do with bugs reported against Rawhide between the Fedora 12 release and the early branch of Fedora 13 under the new 'no frozen Rawhide' system. After some discussion, the group agreed in principle that these bugs should be rebased to Fedora 13, and Adam volunteered to talk to [[User:poelstra|John Poelstra]] and Dave Lawrence about this. | ||
The next QA weekly meeting will be held on 2010-03-01 at 1600 UTC in #fedora-meeting. The next Bugzappers weekly meeting will be held on 2010-03-02 at 1500 UTC in #fedora-meeting. | |||
The next QA weekly meeting will be held on 2010- | |||
<references/> | <references/> | ||
=== Fedora 13 Alpha | === Fedora 13 Alpha validation testing and delay === | ||
It was a busy week for Alpha validation testing for QA and release engineering, with four Alpha release candidate builds being built and (to some extent) tested. As discussed at the meeting, the first release candidate had significant problems rendering it almost unusable. The second fixed some issues but was still had major problems. The third fixed most of the outstanding issues but introduced a regression which broke x86-64 installation, and could also not be installed from the live image. Despite great effort on the part of the release engineering team, It was not possible to build a fourth release candidate in time to be properly tested before the project-wide release readiness meeting, so QA and release engineering had to vote to delay the Alpha release. As announced<ref>http://lists.fedoraproject.org/pipermail/devel/2010-February/131315.html</ref> by [[User:jkeating|Jesse Keating]], the Alpha release would be delayed by one week, but the Beta and Final schedules would not be changed. The fourth release candidate build was announced<ref>http://lists.fedoraproject.org/pipermail/test-announce/2010-February/000030.html</ref> by [[User:Rhe|Rui He]] on 2010-02-26, and validation testing results looked positive at the time of writing. Andre Robatino announced<ref>http://lists.fedoraproject.org/pipermail/test/2010-February/088807.html</ref>DeltaISOs for RC2 -> RC3, and RC3 -> RC4. | |||
<references/> | <references/> | ||
=== | === Dealing with old Target bugs === | ||
[[User: | [[User:Beland|Christopher Beland]] asked<ref>http://lists.fedoraproject.org/pipermail/test/2010-February/088788.html</ref> what should be done with open bugs that were on the Target lists for previous releases. [[User:Adamwill|Adam Williamson]] replied to agree that this was an unsettled question, and pointed out that practically speaking, neither QA nor development teams had paid much attention to the Target lists for several releases. He suggested the system should be either revised or abandoned. | ||
<references/> | <references/> | ||
=== | === Critical Path Wranglers proposal === | ||
[[User:maxamillion|Adam Miller]] presented | [[User:maxamillion|Adam Miller]] presented<ref>http://lists.fedoraproject.org/pipermail/test/2010-February/088824.html</ref> an outline<ref>http://fedoraproject.org/wiki/QA/JoinCriticalPathWranglers:Draft</ref> of a policy for membership of the QA group in FAS (as initially discussed during the weekly meeting). He asked for the group to comment on the proposal and suggest possible refinements and improvements. | ||
<references/> | <references/> | ||
Revision as of 00:54, 2 March 2010
QualityAssurance
In this section, we cover the activities of the QA team[1].
Contributing Writer: Adam Williamson
Test Days
Last week's Test Day[1] was on the langpack plugin for yum[2]. The event suffered a little from falling in the same week as the Alpha candidate testing process, but testing did result in the identification of two bugs. Thanks to those who came out to test.
Next week's Test Day[3] will be on the use of SSSD by default[4]. As the page says, "The prime benefit of the System Security Services Daemon is support for offline logins. Above and beyond the traditional pam_ldap or pam_krb5 approaches, the SSSD would remove the need for laptop users of Fedora to maintain a local account, separate from their centrally-managed account, to work offline or disconnected from the central servers." This is a significant feature for many people, so please come out and help us test it! The Test Day will run all day on Thursday 2010-03-04 in the #fedora-test-day IRC channel.
If you would like to propose a main track Test Day for the Fedora 13 cycle, please contact the QA team via email or IRC, or file a ticket in QA Trac[5].
Weekly meetings
The QA group weekly meeting[1] was held on 2010-02-22. The full logs are available[2]. Adam Williamson noted that the privilege escalation policy[3] the group had helped to create had been approved by FESco and was now an official policy.
James Laska noted that the first Alpha release candidate was now available for testing[4]. Jesse Keating mentioned that it had turned out to be quite broken. James pointed out the test result matrix[5] as the best summary of exactly how broken. The group briefly discussed the causes for the regressions from the second test compose, with the X server input switch from HAL to dbus being one of the major causes, and some errors in anaconda changes the other. Kamil Paral pointed out that an SELinux bug was impeding desktop validation testing. After some discussion, the group agreed testing could go ahead with SELinux disabled and negative results recorded, but not positive results (as it was impossible to be sure if the result would also be positive with SELinux enabled). The group also discussed the best approach to testing to try and get all critical issues fixed without having to delay the Alpha release, and agreed to focus on closing as many open bugs as possible even with the known-bad RC1 images and then testing RC2 quickly once it was made available.
Adam Williamson briefly mentioned that he would be putting the topicof Fedora 13 bug procedures up for discussion at the next day's Bugzappers meeting.
The group discussed the question of membership of the QA group in FAS for a while. This has historically been unimportant as it was not used for anything, but in future, positive feedback in Bodhi for critical path packages will be required from members of QA or Release Engineering before the update can be pushed, so the question of group membership becomes important. Adam Miller volunteered to write an initial draft of a policy / procedure on group membership.
Adam Miller recapped his call for help with QA brainstorming for the new security spin[6]. James Laska pointed out that there had recently been a test day for sectool, and suggested those involved with that may be interested in helping.
The Bugzappers group weekly meeting[7] was held on 2010-02-23. The full log is available[8]. The group discussed Jon Stanley's suggestions[9]. No-one could immediately think of any extra resources the project in general could be providing that would help the Bugzappers' work. The question of sending out updates regarding the project's work became a discussion of the long-pending triage statistics project. Adam Williamson said that Brennan Ashton had been discussing incorporating the project into the Fedora Community system at FUDCon. Edward Kirk said he had blogged in the last couple of weeks about starting work on this. After the meeting, Brennan contacted the group to let them know he was working actively on the Fedora Community system and had been posting about his work on the infrastructure SIG mailing list. He also granted Matej Cepl administrator access to the triage section on fedorahosted.
Adam Williamson introduced the topic of what to do with bugs reported against Rawhide between the Fedora 12 release and the early branch of Fedora 13 under the new 'no frozen Rawhide' system. After some discussion, the group agreed in principle that these bugs should be rebased to Fedora 13, and Adam volunteered to talk to John Poelstra and Dave Lawrence about this.
The next QA weekly meeting will be held on 2010-03-01 at 1600 UTC in #fedora-meeting. The next Bugzappers weekly meeting will be held on 2010-03-02 at 1500 UTC in #fedora-meeting.
- ↑ http://fedoraproject.org/wiki/QA/Meetings
- ↑ http://fedoraproject.org/wiki/QA/Meetings/20100222
- ↑ http://fedoraproject.org/wiki/Privilege_escalation_policy
- ↑ http://lists.fedoraproject.org/pipermail/test-announce/2010-February/000023.html
- ↑ http://fedoraproject.org/wiki/Test_Results:Fedora_13_Alpha_RC1_Install#Test_Matrix
- ↑ http://fedoraproject.org/wiki/SecuritySpin:QA_Brainstorm
- ↑ http://fedoraproject.org/wiki/BugZappers/Meetings
- ↑ http://meetbot.fedoraproject.org/fedora-meeting/2010-02-23/fedora-meeting.2010-02-23-15.04.log.html
- ↑ http://lists.fedoraproject.org/pipermail/test/2010-February/088644.html
Fedora 13 Alpha validation testing and delay
It was a busy week for Alpha validation testing for QA and release engineering, with four Alpha release candidate builds being built and (to some extent) tested. As discussed at the meeting, the first release candidate had significant problems rendering it almost unusable. The second fixed some issues but was still had major problems. The third fixed most of the outstanding issues but introduced a regression which broke x86-64 installation, and could also not be installed from the live image. Despite great effort on the part of the release engineering team, It was not possible to build a fourth release candidate in time to be properly tested before the project-wide release readiness meeting, so QA and release engineering had to vote to delay the Alpha release. As announced[1] by Jesse Keating, the Alpha release would be delayed by one week, but the Beta and Final schedules would not be changed. The fourth release candidate build was announced[2] by Rui He on 2010-02-26, and validation testing results looked positive at the time of writing. Andre Robatino announced[3]DeltaISOs for RC2 -> RC3, and RC3 -> RC4.
Dealing with old Target bugs
Christopher Beland asked[1] what should be done with open bugs that were on the Target lists for previous releases. Adam Williamson replied to agree that this was an unsettled question, and pointed out that practically speaking, neither QA nor development teams had paid much attention to the Target lists for several releases. He suggested the system should be either revised or abandoned.
Critical Path Wranglers proposal
Adam Miller presented[1] an outline[2] of a policy for membership of the QA group in FAS (as initially discussed during the weekly meeting). He asked for the group to comment on the proposal and suggest possible refinements and improvements.