Fedora Testing Meeting - November 16 2006, 2200 UTC, #fedora-testing on freenode
See FedoraTesting for more info about the Fedora Testing project.
Agenda
1. Review from last week a. how-to-test template a. Beaker/RHTS status a. Bugzilla status 1. Fedora Summit recap a. Updates tool and update policy a. BugZappers and community a. Bugzilla improvements
Notes
Raw IRC log: ["Testing/Logs/20061116"]
Test plans
A bit too complicated for most people's liking.
17:08 <@wwoods> I think we're really just going to have something nicely simple for that, like 3 parts 17:08 <@wwoods> Description (an overview of the test plan's purpose) 17:09 <@wwoods> Actions (steps to follow to test some bit of functionality) 17:09 <@wwoods> and Expected Results (duh)
- We can't require tests for all packages built yet, but this is a long-term goal.
- We want people to do their normal exploratory testing, and when they find something interesting or useful, share that:
17:13 < clumens> i imagine that people would regularly do exploratory testing and then once they find something, formalize it so in the future we know what to look for
- Action Item: wwoods to add template to wiki on Nov. 17
- Still need a way to keep track of who is testing packages, and how. Perhaps this could be integrated with the next item.
New Updates System
See ["Infrastructure/UpdatesSystem"] for further details.
- We want testers to use the new Updates System to test Pending Updates (e.g. things in the updates-testing repo)
- Pending updates should have links to how-to-test documents, changelog, automated test results
- Pending updates should have "Love it!/Hate it!" links to allow for feedback from testers
Similar to mugshot: http://developer.mugshot.org/wiki/Image:Love_Hate_Account_Infos.jpg
- "Hate It!" should give the user a list of common/previously used reasons, or "Other"
i. "broken for my hardware", "doesn't start", "missing dependency on X", etc. i. could also have bugs filed against that particular package NVR i. "Other": allow user comment - this comment would go into a bug for that package NVR
Beaker / RHTS Status
See ["QA/Beaker"] for further details.
Public Beaker results coming soon:
17:21 < dmalcolm> I did some hacking on a result browser for Beaker that could sit outside the firewall, called "Honeydew" 17:21 < dmalcolm> but it's too early to be useful (it's in testing.108.redhat.com subversion in the incubator dir)
- Action Item: dmalcolm to mock up XML-RPC interface to allow (e.g.) the Updates System to fire off test runs.
- dmalcolm mentioned rpmlint regression tests
i. is package correctness something that QA should catch, or the package builder / build system automated checks? i. Action Item: wwoods will ask about rpmlint inclusion in ["FedoraSummit/NewBuildSystem"]
Bugzilla Improvements
- Voting on bugs - Love/Hate from the Updates System discussion would use this
- Bugzilla RPG!
GNOME has something like this already (http://davyd.livejournal.com/189538.html). We can't use the GNOME code directly - bugzilla.redhat.com is too different from upstream bugzilla - so we'll need to write an external thing that pulls info out of bugzilla for all our "players".
18:20 < che> wwoods, i have messed with game design in the past :) 18:20 < che> wwoods, not sure how far you wanna go :) 18:20 <@wwoods> che: cooool. I'll try to work on the backend bits - pulling user info for a given bugzilla userid 18:20 <@wwoods> then we start working out how to award XP and letting people choose classes and such 18:20 <@wwoods> heh 18:21 < che> wwoods, something like that.. and having items 18:21 < che> wwoods, maybe we could do something like "bugzilla avatar battle weekends"
i. Give XP for opening bugs, closing bugs, writing test docs, testing packages and reporting results, etc. i. Multiple "classes" to choose from, giving different amusing titles etc. i. Give people items for participating in special Bug Days, etc.
BugZappers and Bug Days
See BugZappers for further details.
- Start doing bug days again! Every Friday, in #fedora-triage!
- Main goal is reporting bugs and cleaning existing bug reports, but with developer help we can try tracing and fixing bugs too
- Writing FAQ pages in the wiki is also very, very helpful
- Have weekly themes! (Or "quests, once we get the bugzilla RPG stuff working). Theme ideas:
Laptop suspend/resume, NetworkManager, SELinux, xorg autoconf, Evolution (and other important apps - Firefox, Totem, etc.)
18:28 < Lovechild> I would love to do intensive bughunts.. one app a day for a week.. 7 major apps on our desktop getting hammered 24/7
- Start announcing Bug Days at the beginning of each week, on fedora-test-list