QA/Meetings/20091019

= Attendees =

People present (lines said)


 * jlaska (174)
 * wwoods (52)
 * Oxf13 (52)
 * adamw (50)
 * kparal (28)
 * Viking-Ice (13)
 * Southern_Gentlem (11)
 * lili_ (10)
 * nirik (10)
 * zodbot (8)
 * mcepl (3)
 * buggbot (2)
 * mbonnet (2)
 * notting (1)
 * Jeff_S (1)

Regrets:
 * He Rui (rhe)

= Agenda =


 * Proposed meeting agenda
 * meetbot summary

Preview Meeting follow-up
QA/Meetings/20091012


 * jlaska will investigate the 5 anaconda F12Beta issues to determine when the failures were introduced

INPRGRESS - Adam Williamson posted feedback during the beta release readiness meeting 36 issues were marked as beta blockers prior to freeze, 16 after the freeze.. I am manually inspecting the bugs added to the blocker list after the freeze (Sept 31) to determine if there is a pattern. So far, only 2 bugs took longer than 2 days to be escalated to the F12Beta blocker list. More manual fiddling required


 * jlaska to investigate packaging of israwhidebroken.com for production instance

Packaged thanks to python setuptools. Need to determine whether to incorporate into autoqa now. Wwoods had the suggestion of creating a autoqa/front-ends directory to include this and other "Is broken?" front-ends.


 * jlaska to request autoqa-devel@ for autoqa development discussion and patch review

https://fedorahosted.org/fedora-infrastructure/ticket/1733

Beta test review
Jlaska asked the group for feedback on what worked and what needs improvement for how the team tracks blocker bugs and organizes testing.

Liam suggested on coming up with ways to achieve 100% test execution on the test matrix. At least aiming for completion of tier#2 tests in the test plan. Southern_Gentlem suggested looking at the spins test matrix. Jlaska asked if any optimizations are available in the current test matrix, perhaps removing duplicate testing or removing tests altogether. Lili and Jeff_S entertained whether or not that would mean reduced quality in the end. Jlaska asked if the two test matrices could combine efforts somehow?

Adamw noted that while the results weren't tracked in the wiki matrix, he has the triumvirate of video adapters (radeon, nvidia and intel). Each adapters was verified against the beta. As for the test matrix, Adam suggested it can be overwhelming at first. Perhaps re-ordering the links such that the UNTESTED cells are at the top of the page? Jlaska came back to reality after thinking about a possible crowd-sourcing option for gathering install use case feedback from testers.

This initiated a brief discussion by Oxf13 about a formal test case management system (TCMS). Jlaska suggested that a TCMS isn't something we'd really want or need in QA at this time. Adam added, yes, if only we had a TCMS we could spend lots of time maintaining and debugging it instead of testing things! Jlaska concluded by adding, I'm really not convinced that a TCMS created and maintained by this small group is the way to improving quality in Fedora.

Summary of recommendations:
 * 1) integration between lili and Southern_Gentlem's test matrices
 * 2) making it easier to see what testing remains in lili's test matrix

AutoQA Updates from wwoods and kparal
Wwoods and kparal provided updates on autoqa progress.


 * we're still working on getting the production autoqa instance active
 * Hardware shipment to PHX2 still inprogress
 * More packaging work needed to integrate israwhidebroken.com
 * I've given up on patching autotest to add info about the job ID to the tests as they run. far simpler just to write a small "how to find test results" document rather than maintaining an invasive and complicated patch
 * A draft koji hook/watch has been pushed to git (see https://fedorahosted.org/autoqa/browser/hooks/post-koji-build/watch-koji-builds.py). it works basically like the other watchers, listing all new builds since the previous run.  Next step is to define what any koji-watcher test would need as arguments/requirements.  From there, we can complete the hook.

rpmguard
Kparal provided updates on the packagediff/rpmguard front:
 * name: the script for displaying important differences between rpms, originally called 'packagediff', was renamed to 'rpmguard'
 * location: the autoqa project has provided hosting for the rpmguard (see http://git.fedorahosted.org/git/autoqa.git?a=tree;f=tests/rpmguard)
 * Kparal noted he has several test packages built using rpmfluff to test rpmguard functionality. So far so good, but he welcomes any community testing.

Kparal asked for community feedback on 2 aspects of rpmguard:
 * 1) testing - I will be very glad if you download it, try it, provide a feedback on it. But please be sure also to use latest rpmdiff from svn (prerequisite for rpmguard), because there were some important bugs fixed recently. I will also write a blogpost and post a message to mailing lists in the next days.
 * 2) output - Also, if you have some suggestions how to restructure the output of rpmguard, to be better parseable/machine processable, let me know. Especially if wwoods have some requirements what to ouput so that autoqa can utilize it, I'm all ears.

How_to_Debug_ Update from Viking-Ice
VIking-Ice indicated adamw was looking into testing a new wiki-style Template. Adamw expertly delegated the to rjune's expertise, which resulted in 2 new pages:
 * https://fedoraproject.org/wiki/How_to_debug_Dracut_problems2
 * https://fedoraproject.org/wiki/Template:How_to_debug2

Adamw indicated that we should be able to come up with a template which defines the general layout in a flexible enough way that we can still customize individual pages. However, some additional work is needed. If we cannot produce a page in a reasonable time frame, we will continue manual wiki edits.

Adamw asked if someone would like to rename the existing Category:Debugging pages to use the new format: How to debug problems. Adamw also suggested changing the back-links to avoid double redirect issues on the wiki. Jlaska agreed to rename the pages.

CritPath testing
Oxf13 asked for extra eyes/fingers as crit-path tag requests come in for final.

Adamw asked whether the process for requesting a post-Beta tag was documented. To track crit-path updates, F13 recommended:
 * 1) subscribe to rel-eng@lists.fedorahosted.org list
 * 2) monitor RSS feed - https://fedorahosted.org/rel-eng/timeline?ticket=on&changeset=on&milestone=on&wiki=on&max=50&daysback=90&format=rss (may require URL tweakage)

Jlaska asked if there was a standard set of information tagging requests should include? For example, a link to the F12Blocker bug, a link to the patch(es), and a statement of severity or user impact. F13 advised there is no standard at this time, but we do currently ask:
 * 1) what they are fixing,
 * 2) why,
 * 3) what testing they've done,
 * 4) and what happens if we don't include it

Jlaska asked if a template exists for this information? F13 noted that the make tag-request process asks for this information. Mcepl provided a link to more details on this process at ReleaseEngineering/FinalFreezePolicy.

Common_F12_Bugs
Adamw reminded the group, if you're aware of any bug that's in the released beta images that you think should be documented there, please either add it (there's guidelines on layout in the wiki page source) or add the 'CommonBugs' keyword to the bug report.


 * Bugs that need Common_F12_Bugs documentation

rats_install still failing
Jlaska asked if there were any updates on the download.fedora.devel.redhat.com rsync from download.fedoraproject.org. It appears autoqa rats_install results are still failing due to bad content on the download site (see https://fedorahosted.org/pipermail/autoqa-results/2009-October/001654.html).

Still needs investigation.

F-12 Talking Awesomeness talking points
Adamw recommended F12_Beta_Announcement for folks looking for talking points about what's new in Fedora 12.

Upcoming QA events

 * 2009-10-12 - F12 Beta go/no_go meeting (@ 18:00pm - rel-eng meeting)
 * 2009-10-13 - BugZappers/Meetings
 * 2009-10-14 - Beta Project Wide Release Readiness Meeting
 * 2009-10-15 - i18n Test Day

= Action items =


 * jlaska to rename other debug pages (see https://fedoraproject.org/wiki/Category:Debugging) to be consistent with "How to debug problems". Update back-links also

= IRC transcript =

Generated by irclog2html.py 2.7 by [mailto:marius@pov.lt Marius Gedminas] - find it at mg.pov.lt!