QA/Logs/20061207

--- Log opened Thu Dec 07 12:03:44 2006 12:03 <@wwoods> meeting time! waiting a few minutes for people to trickle in. 12:04 -!- wwoods changed the topic of #fedora-testing to: Fedora QA | Meeting Dec 7 - see http://fedoraproject.org/wiki/QA/Meetings/20061207 12:07 [Users #fedora-testing] 12:07 [@ChanServ] [ dmalcolm]  [ muep   ]  [ Sonar_Guy] 12:07 [@wwoods ]  [ ixs     ]  [ poelcat]  [ thl      ] 12:07 -!- Irssi: #fedora-testing: Total of 8 nicks [2 ops, 0 halfops, 0 voices, 6 normal] 12:07 -!- rdieter [n=rdieter@sting.unl.edu] has joined #fedora-testing 12:08 -!- daMaestro [n=jon@fedora/damaestro] has joined #fedora-testing 12:08 * daMaestro lurks 12:09 <@wwoods> grace period over, meeting starts... NOW 12:09 <@wwoods> Hello and welcome and all that! The agenda (if you haven't noticed already) is in the topic: http://fedoraproject.org/wiki/QA/Meetings/20061207 12:10 <@wwoods> (Yeah, I'm calling it Fedora QA now. I want people to know we have a QA team. More about that later) 12:10 <@wwoods> Okay, first, review from last time: sorry it's been a while, but there was the US Thanksgiving holiday and such. 12:11 <@wwoods> We've got a simple template for documenting how you test a package: http://fedoraproject.org/wiki/QA/HowToTestTemplate 12:12 nice! 12:12 <@wwoods> once the fedora-updates-tool is running, each item in the Pending Updates list should point to the HowToTest for that package 12:12 HowToTest$PACKAGE ? 12:12 <@wwoods> yeah, I think that'd be best 12:13 <@wwoods> maybe HowToTest/$PACKAGE 12:13 <@wwoods> or QA/HowToTest/$PACKAGE 12:13 What about e.g. the various families of packages e.g. xorg-x11-drv-$FOO 12:13 <@wwoods> or am I trying to be too clever? 12:14 <@wwoods> dmalcolm: hmm, good question. I think the drivers are actually their own packages these days 12:14 actually, it may make sense even for those packages, as people could e.g. say that they use a particular driver on that driver's page 12:14 <@wwoods> but for things like openssh-{client,server,askpass,...} 12:14 Is $PACKAGE the SRPM, or the RPM? 12:15 <@wwoods> it will be the SRPM 12:15 * dmalcolm pokes holes 12:15 <@wwoods> but! people are welcomed and encouraged to write separate docs for the subpackages 12:15 <@wwoods> you *could* cram all of the openssh testing docs into the openssh page, or you could have that page point to the individual openssh-client, openssh-server, ... pages 12:16 <@wwoods> just like writing any kind of wikified/HTML page, you gotta decide when a page gets too big and when you will split it into subpages 12:16 <@wwoods> but as far as the updates system is concerned, there should be one wiki page per SRPM 12:17 <@wwoods> and each update will have a link to that wiki page. whether that wiki page has instructions or links to other pages with instructions is up to the author, I guess 12:18 <@wwoods> and someday when we get the Bugzilla RPG going, you'll get XP for writing these docs. natually. 12:18 so - since the build/release process is focusseed around SRPMs, so the testing process should be? 12:18 <@wwoods> for updates, yes 12:19 <@wwoods> when we get closer to FC7t1, we should start on functional testing plans 12:19 <@wwoods> e.g. "how to test Xen" 12:19 <@wwoods> but for the next couple of months the focus should be on the fedora-updates-system 12:20 <@wwoods> (which is analagous to the RHEL Errata tool, for the rhatters) 12:21 <@wwoods> other stuff from the last meeting: I was supposed to ask about getting rpmlint into the new build system. Haven't done that. 12:21 <@wwoods> Not sure if there's been much movement on the build system, though. 12:22 <@wwoods> dmalcolm: you mentioned something about an XMLRPC interface for firing off test runs? 12:22 yeah, I recently had a long chat with lmacken about the Fedora Update system 12:22 <@wwoods> oh did you? awesome. 12:22 we started mapping out how some of the interfaces might work 12:23 but I haven't had time to go and implement it yet, alas 12:23 <@wwoods> I think that's really where we need to be focusing for the next few months 12:23 <@wwoods> that's fine.. keep it in mind. I'm probably going to be bothering lmacken about the updates tool a lot 12:24 <@wwoods> trying to add stuff like discussed last time 12:24 <@wwoods> I think that's where the RHTS stuff will fit, as well 12:24 <@wwoods> (moving on to 2.) 12:25 <@wwoods> when a new update is added, it should probably have a [run automated tests] button 12:25 so the vision is: when a SRPM candidate for a Fedora Update is suggested by a Core/Extras/Freedom/whatever-it's-called person, we'll queue up a bank of tests on the before and on the after trees 12:25 <@wwoods> or a [no automated tests found! help write some!] link to RHTS docs 12:25 and integrate the test results into the notification email that goes to the fedora-test mailing list 12:26 wwoods: good idea! 12:26 does this sound sane? 12:26 <@wwoods> hmm, yes 12:26 <@wwoods> so maybe instead, it will either have a link to the results 12:26 <@wwoods> or a link to documentation on how to write tests 12:26 <@wwoods> (And a [run tests] button to re-run them after you write them) 12:27 <@wwoods> I think that'd be fantastaic. 12:28 <@wwoods> fantastic even. 12:28 <@wwoods> dmalcolm, poelcat: any new RHTS stuff since the last meeting? 12:28 Do we have an overall picture or design somewhere that shows how all these pieces might go together? 12:29 <@wwoods> poelcat: no, but we could certainly make one. I think it'd be a good idea to add that to the UpdatesSystem docs. 12:29 wwoods: i'd be glad to write it up or draw it 12:29 if you can point me to all the people and pieces :) 12:30 <@wwoods> I was going to try to mock up these bits in the fedora-updates-tool code, but it's proving tricky 12:30 wwoods: are you using the old or the new code? 12:30 * poelcat likes to have a high level design first 12:30 <@wwoods> poelcat: cool. I'd suggest starting with http://fedoraproject.org/wiki/Infrastructure/UpdatesSystem 12:31 <@wwoods> as for people to talk to: myself, lmacken, jkeating if you need info about the build system 12:32 <@wwoods> also see the discussion of the Updates System from the previous meeting (http://fedoraproject.org/wiki/Testing/Meetings/20061116) 12:35 <@wwoods> if there's no further discussion for RHTS (and we aren't about to have a netsplit).. 12:35 <@wwoods> 3. Official QA group! 12:36 <@wwoods> do we want to restrict access to the updates system to official QA members? 12:36 errr... are we having an official QA group? 12:36 <@wwoods> that's the question 12:36 <@wwoods> heh 12:37 <@wwoods> I'm not sure if it's useful or necessary, but it'd be nice for restricting access to QA tools 12:37 <@wwoods> (Beaker etc.) 12:37 <@wwoods> also it's good to have a roster of people so you can estimate manpower and such 12:38 <@wwoods> would we rather let anyone who feels like it help out? 12:39 <@wwoods> oh - QA group membership would also mean having a character in the (theoretical) Bugzilla RPG 12:39 <@wwoods> so. is this necessary? opinions? 12:40 K - so I can see there being a need for some access control e.g. for the test lab 12:40 s/K/OK/ 12:40 since someone malicious could tie it up by constantly running intensive jobs, spamming the queue 12:41 <@wwoods> right 12:41 I thought we' 12:41 <@wwoods> and someday I'd like to be able to let (trusted) testers reject a pending update 12:41 I thought we'd want to open up the RPG to everyone working in bugzilla on Fedora 12:42 <@wwoods> dmalcolm: yes, but QA group membership would give you bug group membership and a bugzilla RPG account, among other things. probably edit access to parts of the wiki, for test docs 12:42 <@wwoods> but that's a sufficient (not necessary) condition 12:44 <@wwoods> anyway, I think maybe this stuff isn't necessary until we have things we need to restrict access to 12:44 <@wwoods> restrict / give 12:45 <@wwoods> (translation: I'm lazy and I'll wait until it's needed) 12:45 heh 12:46 <@wwoods> the second part though - writing "how to be a tester" guides - is still very helpful 12:47 <@wwoods> that's another rainy-day project that I'll get to shortly after I write an xorg driver for the Wii remote 12:47 <@wwoods> er, I mean.. ASAP 12:47 <@wwoods> anyway, bugzilla status: no work on bugzilla RPG yet, but there's a bugzilla bot in #fedorabot 12:47 <@wwoods> (can't believe I didn't know it was there!) 12:48 <@wwoods> it's proven immensely helpful for me to help keep bugzilla clean. highly recommended. 12:48 <@wwoods> (p.s. users who submit the same bug 7 times in a row get furious frowning-at) 12:49 what does RPG stand for? 12:49 <@wwoods> Role Playing Game! 12:50 oh, the game theory idea as it relates to community participation? 12:50 <@wwoods> that's something I should mock-up ASAP. mostly because it's fun. 12:50 * dmalcolm is a 13th Level Bug Triager on gnome.org 12:50 <@wwoods> yeah 12:50 <@wwoods> I need to work out a frontend for it and data sources for granting points/XP 12:51 wwoods: are there examples of other projets besides gnome that have done this successfully? 12:51 poelcat: http://bugzilla.gnome.org/page.cgi?id=points.html 12:51 <@wwoods> poelcat: I don't know any others off the top of my head, but I'll look into it 12:52 the formula bugzilla.gnome.org uses is designed to favor triage of bugs - you get more points for closing a bug than for reporting one 12:53 this was by design: they had no shortage of good bugs, but needed people to triage them 12:53 <@wwoods> right, I suspect we should award a small number of points for giving feedback on a pending update, more for reporting, more still for triaging, more still for writing howtotest docs, etc. 12:53 http://bugzilla.gnome.org/reports/points.cgi 12:55 <@wwoods> probably we'll have to have a beta period to help work out the weighting and the various factors 12:55 <@wwoods> once we get hooks in place to *read* those factors, anyway 12:55 (gnome uses logarithms, rather than just scaling) 12:56 <@wwoods> hm. something to keep in mind, then. 12:56 <@wwoods> okay, anyway, a quick note - there's a lot of bugs being reported about yum segfaulting - the canonical bug for this is https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=213963 12:56 <@wwoods> note how many duplicates there are. this gets reported a *lot* 12:57 <@wwoods> I should think about putting it on the Common Bugs page. 12:57 <@wwoods> I'll have to talk to jeremy et. al. about workarounds and the ETA on a fix 12:58 <@wwoods> but, yeah, something to watch out for in your triaging efforts. 12:58 <@wwoods> And that's about it for the main agenda. Is there anything that could use further discussion? 13:01 <@wwoods> okay then! meeting over! bang the gavels! 13:01 <@wwoods> I'll post a log and meeting notes as soon as I can. 13:02 <@wwoods> thanks, guys. --- Log closed Thu Dec 07 13:02:11 2006