FWN/Beats/QualityAssurance

QualityAssurance

In this section, we cover the activities of the QA team.

Contributing Writer: Adam Williamson

Test Days

This week's regular test day was on the rewritten handling of storage devices in Anaconda. Dave Lehman, Chris Lumens and Joel Granados were the developers present. Several people showed up and provided valuable testing in a wide range of scenarios, and the developers were able to identify and resolve several bugs. Further testing in this area is still very helpful. The Wiki page contains instructions for using a supplementary image while installing Rawhide, to use the new storage code successfully, but the code will soon be available directly in Rawhide, so testing can be performed simply by attempting to install Rawhide in as many different storage scenarios as possible.

Next week's test day is tentatively scheduled for testing Intel graphics devices, especially the new kernel mode setting support and identifying performance regressions from Fedora 10. It will be held on Thursday (2009-03-12) in the #fedora-qa channel on Freenode IRC. If you use an Intel graphics card, please come by to help make sure it will be well-supported in Fedora 11 - the more testing, the better the code!

Weekly meetings

The QA group weekly meeting was held on 2009-03-04. The full log is available. Will Woods pointed out that the next week's meeting would be an hour earlier for most people, after the onset of Daylight Savings Time. Adam Williamson and Jesse Keating, as the resident West Coast representatives, led a revolt against having to wake up at 7 a.m., and the group agreed to move the meetings to 1700UTC from 2009-03-11.

James Laska reported on the progress of the project to make the Semantic test result plugin for mediawiki available. He reported that he is currently trying to make the plugin work in his test setup prior to building a Fedora package for it, as the infrastructure group requires all software used for Fedora systems be available as a Fedora package. Adam Williamson offered to help with the packaging.

Will Woods and Jesse Keating reported that there had been no progress on the autoqa systems this week, as Jesse had been tied up doing the mass rebuild of Rawhide.

Will Woods reported that 33 of the Fedora 11 proposed features have not yet been reviewed by the QA group to ensure that they include a workable test plan, and appealed for help from the group in getting this process completed. He noted that features for which test days have already taken place are likely to have workable test plans in place, as these are generally necessary for a test day to happen, so suggested at least those features could be quickly reviewed and, most likely, approved.

Adam Williamson reported that the test day for the new nouveau driver for NVIDIA hardware was already planned and prepared, but that he was waiting on developer replies for the planned Intel KMS and automatic fonts and MIME installer test days. Jóhann Guðmundsson suggested having a single big graphics test day, but Adam Williamson explained that he did not want to do that as it would be too large and unmanageable. James Laska suggested running the Piglit OpenGL test suite as part of the test days for drivers with usable 3D support (radeon and intel).

Jesse Keating reported that the mass rebuild of Rawhide was complete. Adam Williamson pointed out that three large bugs had resulted from GCC 4.4 optimization problems after the rebuild, and Will Woods reported that this had been discussed during the release engineering meeting. Will noted that the new hashing system in RPM was not backwards compatible, the upshot being that those upgrading from Fedora 11 Alpha to current Rawhide need to run 'yum update rpm' first. He queried why yum did not automatically update itself and rpm before other packages, and Jesse Keating explained it was because this would usually bring in hundreds of other packages via Python and glibc dependencies in any case, and so was not worth the effort.

Jesse Keating reported that live CD image builds from current Rawhide were not working very well, and Anaconda is often broken while its developers are busy working on the storage rewrite and EFI features.

The Bugzappers group weekly meeting was held on 2009-03-03. The full log is available. The group agreed to make sure the two most important Greasemonkey scripts for triagers were easily available in a central place, and this was implemented by making them available directly from the Wiki Tools page.

The group discussed Christopher Beland's plan to re-organize the Wiki area. It was agreed that a mailing list discussion should take place to create and agree upon a new front page for the Bugzappers wiki area, and work could then progress on re-writing and re-arranging other pages based on the organization system set up by the new front page. The group also discussed and agreed upon a plan for revising the Components page, and Christopher Beland pointed out that the current bug flow diagram is incorrect, as it dates from before NEEDINFO was converted from a bug status into a flag. Edward Kirk bravely volunteered to fix the picture.

Finally, the group discussed creating SOPs (Standard Operating Procedures) for Bugzappers, along the lines of those already used by the Infrastructure group. John Poelstra proposed the first SOP cover the procedure for gaining membership in the Bugzappers, and further proposed that it should involve the prospective new member posting a self-introduction email to the mailing list. Christopher Beland and Edward Kirk opposed this as they were worried that some new members would not feel comfortable posting such a message, particularly if it contained personal information. The group agreed to discuss the proposal further on the mailing list.

The next QA weekly meeting will be held on 2009-03-11 at 1700 UTC (note changed time, in UTC reference frame) in #fedora-meeting, and the next Bugzappers weekly meeting on 2009-03-10 at 1700 UTC in #fedora-meeting.

Bugzappers Wiki reorganization

Christopher Beland made several proposals on reorganizing the Bugzappers wiki area. This prompted a long discussion. In the end, Christopher was asked to provide drafts for several of his proposed changes for the group to evaluate. Both Christopher and Edward Kirk provided drafts for a new front page. Adam Williamson commented that both drafts had good elements, and offered to create a new draft to try and combine the two.

20 Second Boot test day follow up

Harald Hoyer posted a follow-up email on the previously-held 20 Second Boot test day, pointing to a blog post where he summarized all the useful data he was able to get from the test day.

Bugzappers meeting schedule

Lalit Dhiri proposed having a second meeting to accommodate those whose schedules made it impossible for them to attend the regular group meetings. Adam Williamson said that was not likely to be practical, but suggested that the meeting time could be moved if a time when more group members would be able to attend could be identified. Susan Lauber suggested using the Wiki's meeting matrix template to handle registering who is available when, and set up a Wiki page for the purpose.

Ubuntu triage discussion

Paul Frields pointed out a long discussion on the topic of bug triaging in Ubuntu, and wondered what the lessons for the Bugzappers might be. Adam Williamson suggested that it showed it is important for triagers to follow up on bugs they triage, rather than just touching them once and then never returning, which can leave the reporter more frustrated than if the bug had never been triaged at all. In the ensuing discussion, John Summerfield suggested that triagers should try to cover one area of which they had substantial knowledge rather than attempting to cover all bugs in all components, and that the Bugzappers group should remember actively to involve package maintainers in the triaging process. Kevin Kofler explained that, within the KDE SIG, maintainers and triagers do work together and communicate via IRC.

Introduction emails

Adam Williamson mooted the proposal from the weekly meeting that introduction emails be required for new group members. John Poelstra supported the proposal, as did Edward Kirk. Christopher Beland suggestedthat anyone who became active on the mailing list but did not write a formal self-introduction email should also be accepted.