QA/Logs/20061207

--- Log opened Thu Dec 07 12:03:44 2006
12:03 <@wwoods> meeting time! waiting a few minutes for people to trickle in.
12:04 -!- wwoods changed the topic of #fedora-testing to: Fedora QA | Meeting Dec 7 - see http://fedoraproject.org/wiki/QA/Meetings/20061207
12:07 [Users #fedora-testing] 
12:07 [@ChanServ]  [ dmalcolm]  [ muep   ]  [ Sonar_Guy] 
12:07 [@wwoods  ]  [ ixs     ]  [ poelcat]  [ thl      ] 
12:07 -!- Irssi: #fedora-testing: Total of 8 nicks [2 ops, 0 halfops, 0 voices, 6 normal] 
12:07 -!- rdieter [n=rdieter@sting.unl.edu]  has joined #fedora-testing
12:08 -!- daMaestro [n=jon@fedora/damaestro]  has joined #fedora-testing
12:08  * daMaestro lurks
12:09 <@wwoods> grace period over, meeting starts... NOW
12:09 <@wwoods> Hello and welcome and all that! The agenda (if you haven't noticed already) is in the topic: http://fedoraproject.org/wiki/QA/Meetings/20061207
12:10 <@wwoods> (Yeah, I'm calling it Fedora QA now. I want people to know we have a QA team. More about that later)
12:10 <@wwoods> Okay, first, review from last time: sorry it's been a while, but there was the US Thanksgiving holiday and such.
12:11 <@wwoods> We've got a simple template for documenting how you test a package: http://fedoraproject.org/wiki/QA/HowToTestTemplate
12:12 < dmalcolm> nice!
12:12 <@wwoods> once the fedora-updates-tool is running, each item in the Pending Updates list should point to the HowToTest for that package
12:12 < dmalcolm> HowToTest$PACKAGE ?
12:12 <@wwoods> yeah, I think that'd be best
12:13 <@wwoods> maybe HowToTest/$PACKAGE
12:13 <@wwoods> or QA/HowToTest/$PACKAGE
12:13 < dmalcolm> What about e.g. the various families of packages e.g. xorg-x11-drv-$FOO
12:13 <@wwoods> or am I trying to be too clever?
12:14 <@wwoods> dmalcolm: hmm, good question. I think the drivers are actually their own packages these days
12:14 < dmalcolm> actually, it may make sense even for those packages, as people could e.g. say that they use a particular driver on that driver's page
12:14 <@wwoods> but for things like openssh-{client,server,askpass,...}
12:14 < dmalcolm> Is $PACKAGE the SRPM, or the RPM?
12:15 <@wwoods> it will be the SRPM
12:15  * dmalcolm pokes holes
12:15 <@wwoods> but! people are welcomed and encouraged to write separate docs for the subpackages
12:15 <@wwoods> you *could* cram all of the openssh testing docs into the openssh page, or you could have that page point to the individual openssh-client, openssh-server, ... pages
12:16 <@wwoods> just like writing any kind of wikified/HTML page, you gotta decide when a page gets too big and when you will split it into subpages
12:16 <@wwoods> but as far as the updates system is concerned, there should be one wiki page per SRPM
12:17 <@wwoods> and each update will have a link to that wiki page. whether that wiki page has instructions or links to other pages with instructions is up to the author, I guess
12:18 <@wwoods> and someday when we get the Bugzilla RPG going, you'll get XP for writing these docs. natually.
12:18 < dmalcolm> so - since the build/release process is focusseed around SRPMs, so the testing process should be?
12:18 <@wwoods> for updates, yes
12:19 <@wwoods> when we get closer to FC7t1, we should start on functional testing plans
12:19 <@wwoods> e.g. "how to test Xen"
12:19 <@wwoods> but for the next couple of months the focus should be on the fedora-updates-system
12:20 <@wwoods> (which is analagous to the RHEL Errata tool, for the rhatters)
12:21 <@wwoods> other stuff from the last meeting: I was supposed to ask about getting rpmlint into the new build system. Haven't done that.
12:21 <@wwoods> Not sure if there's been much movement on the build system, though.
12:22 <@wwoods> dmalcolm: you mentioned something about an XMLRPC interface for firing off test runs?
12:22 < dmalcolm> yeah, I recently had a long chat with lmacken about the Fedora Update system
12:22 <@wwoods> oh did you? awesome.
12:22 < dmalcolm> we started mapping out how some of the interfaces might work
12:23 < dmalcolm> but I haven't had time to go and implement it yet, alas
12:23 <@wwoods> I think that's really where we need to be focusing for the next few months
12:23 <@wwoods> that's fine.. keep it in mind. I'm probably going to be bothering lmacken about the updates tool a lot
12:24 <@wwoods> trying to add stuff like discussed last time
12:24 <@wwoods> I think that's where the RHTS stuff will fit, as well
12:24 <@wwoods> (moving on to 2.)
12:25 <@wwoods> when a new update is added, it should probably have a [run automated tests]  button
12:25 < dmalcolm> so the vision is:  when a SRPM candidate for a Fedora Update is suggested by a Core/Extras/Freedom/whatever-it's-called person, we'll queue up a bank of tests on the before and on the after trees
12:25 <@wwoods> or a [no automated tests found! help write some!]  link to RHTS docs
12:25 < dmalcolm> and integrate the test results into the notification email that goes to the fedora-test mailing list
12:26 < dmalcolm> wwoods: good idea!
12:26 < dmalcolm> does this sound sane?
12:26 <@wwoods> hmm, yes
12:26 <@wwoods> so maybe instead, it will either have a link to the results
12:26 <@wwoods> or a link to documentation on how to write tests
12:26 <@wwoods> (And a [run tests]  button to re-run them after you write them)
12:27 <@wwoods> I think that'd be fantastaic.
12:28 <@wwoods> fantastic even.
12:28 <@wwoods> dmalcolm, poelcat: any new RHTS stuff since the last meeting?
12:28 < poelcat> Do we have an overall picture or design somewhere that shows how all these pieces might go together?
12:29 <@wwoods> poelcat: no, but we could certainly make one. I think it'd be a good idea to add that to the UpdatesSystem docs.
12:29 < poelcat> wwoods: i'd be glad to write it up or draw it
12:29 < poelcat> if you can point me to all the people and pieces :)
12:30 <@wwoods> I was going to try to mock up these bits in the fedora-updates-tool code, but it's proving tricky
12:30 < dmalcolm> wwoods: are you using the old or the new code?
12:30  * poelcat likes to have a high level design first
12:30 <@wwoods> poelcat: cool. I'd suggest starting with http://fedoraproject.org/wiki/Infrastructure/UpdatesSystem
12:31 <@wwoods> as for people to talk to: myself, lmacken, jkeating if you need info about the build system
12:32 <@wwoods> also see the discussion of the Updates System from the previous meeting (http://fedoraproject.org/wiki/Testing/Meetings/20061116)
12:35 <@wwoods> if there's no further discussion for RHTS (and we aren't about to have a netsplit)..
12:35 <@wwoods> 3. Official QA group!
12:36 <@wwoods> do we want to restrict access to the updates system to official QA members?
12:36 < dmalcolm> errr...  are we having an official QA group?
12:36 <@wwoods> that's the question
12:36 <@wwoods> heh
12:37 <@wwoods> I'm not sure if it's useful or necessary, but it'd be nice for restricting access to QA tools
12:37 <@wwoods> (Beaker etc.)
12:37 <@wwoods> also it's good to have a roster of people so you can estimate manpower and such
12:38 <@wwoods> would we rather let anyone who feels like it help out?
12:39 <@wwoods> oh - QA group membership would also mean having a character in the (theoretical) Bugzilla RPG
12:39 <@wwoods> so. is this necessary? opinions?
12:40 < dmalcolm> K - so I can see there being a need for some access control e.g. for the test lab
12:40 < dmalcolm> s/K/OK/
12:40 < dmalcolm> since someone malicious could tie it up by constantly running intensive jobs, spamming the queue
12:41 <@wwoods> right
12:41 < dmalcolm> I thought we'
12:41 <@wwoods> and someday I'd like to be able to let (trusted) testers reject a pending update
12:41 < dmalcolm> I thought we'd want to open up the RPG to everyone working in bugzilla on Fedora
12:42 <@wwoods> dmalcolm: yes, but QA group membership would give you bug group membership and a bugzilla RPG account, among other things. probably edit access to parts of the wiki, for test docs
12:42 <@wwoods> but that's a sufficient (not necessary) condition
12:44 <@wwoods> anyway, I think maybe this stuff isn't necessary until we have things we need to restrict access to
12:44 <@wwoods> restrict / give
12:45 <@wwoods> (translation: I'm lazy and I'll wait until it's needed)
12:45 < dmalcolm> heh
12:46 <@wwoods> the second part though - writing "how to be a tester" guides - is still very helpful
12:47 <@wwoods> that's another rainy-day project that I'll get to shortly after I write an xorg driver for the Wii remote
12:47 <@wwoods> er, I mean.. ASAP
12:47 <@wwoods> anyway, bugzilla status: no work on bugzilla RPG yet, but there's a bugzilla bot in #fedorabot
12:47 <@wwoods> (can't believe I didn't know it was there!)
12:48 <@wwoods> it's proven immensely helpful for me to help keep bugzilla clean. highly recommended.
12:48 <@wwoods> (p.s. users who submit the same bug 7 times in a row get furious frowning-at)
12:49 < poelcat> what does RPG stand for?
12:49 <@wwoods> Role Playing Game!
12:50 < poelcat> oh, the game theory idea as it relates to community participation?
12:50 <@wwoods> that's something I should mock-up ASAP. mostly because it's fun.
12:50  * dmalcolm is a 13th Level Bug Triager on gnome.org
12:50 <@wwoods> yeah
12:50 <@wwoods> I need to work out a frontend for it and data sources for granting points/XP
12:51 < poelcat> wwoods: are there examples of other projets besides gnome that have done this successfully?
12:51 < dmalcolm> poelcat: http://bugzilla.gnome.org/page.cgi?id=points.html
12:51 <@wwoods> poelcat: I don't know any others off the top of my head, but I'll look into it
12:52 < dmalcolm> the formula bugzilla.gnome.org uses is designed to favor triage of bugs - you get more points for closing a bug than for reporting one
12:53 < dmalcolm> this was by design: they had no shortage of good bugs, but needed people to triage them
12:53 <@wwoods> right, I suspect we should award a small number of points for giving feedback on a pending update, more for reporting, more still for triaging, more still for writing howtotest docs, etc.
12:53 < dmalcolm> http://bugzilla.gnome.org/reports/points.cgi
12:55 <@wwoods> probably we'll have to have a beta period to help work out the weighting and the various factors
12:55 <@wwoods> once we get hooks in place to *read* those factors, anyway
12:55 < dmalcolm> (gnome uses logarithms, rather than just scaling)
12:56 <@wwoods> hm. something to keep in mind, then.
12:56 <@wwoods> okay, anyway, a quick note - there's a lot of bugs being reported about yum segfaulting - the canonical bug for this is https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=213963
12:56 <@wwoods> note how many duplicates there are. this gets reported a *lot*
12:57 <@wwoods> I should think about putting it on the Common Bugs page.
12:57 <@wwoods> I'll have to talk to jeremy et. al. about workarounds and the ETA on a fix
12:58 <@wwoods> but, yeah, something to watch out for in your triaging efforts.
12:58 <@wwoods> And that's about it for the main agenda. Is there anything that could use further discussion?
13:01 <@wwoods> okay then! meeting over! bang the gavels!
13:01 <@wwoods> I'll post a log and meeting notes as soon as I can.
13:02 <@wwoods> thanks, guys.
--- Log closed Thu Dec 07 13:02:11 2006