QA/Meetings/20101206

From FedoraProject

< QA | Meetings
Revision as of 17:31, 6 December 2010 by Jlaska (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Contents

Attendees

People present (lines said)

  1. jlaska (153)
  2. adamw (58)
  3. kparal (26)
  4. pjones (23)
  5. nirik (13)
  6. mdomsch2 (9)
  7. mkrizek (6)
  8. fenrus02 (3)
  9. Viking-Ice (3)
  10. jsmith (3)
  11. dgilmore (3)
  12. abadger1999 (1)

Unable to attend:

  1. Rhe (hopefully sleeping)

Agenda

Previous meeting follow-up

  1. jlaska to prep F15 test day wiki and request ideas on test@
  2. Nirik to start discussion on test list related to updates ideas (see http://lists.fedoraproject.org/pipermail/test/2010-November/095756.html)

Many ways to get involved in QA ...

Call for Test Days

Owner 
jlaska
Summary
Call for ideas/owners for Fedora 15 QA/Test_Days
Monitor list of approved features for ideas
AdamW initiated GNOME3 discussion on desktop@ list
Next steps ...
Contribute ideas to Talk:QA/Fedora_15_test_days
Identify owners to organize and host test days

Python script communicating with bugzilla

Owner 
Bruno
Tune existing python scripts to detect UNTESTED blocker bugs
Next steps ...
See fedora-qa ticket#89

Requirements review for Fedora test case management

Owner 
rhe
Requirements review for Fedora test case management
Next steps ...
See fedora-qa ticket#152

Critical Path test case development

Owner 
adamwill
Critical Path test case development
Next steps ...
See fedora-qa ticket#154

AutoQA Update

Owner 
kparal
Summary
  1. All autoqa config files are now copied from the server to the client when executing tests
  2. Watchers and autoqa script now return non-zero return code when test scheduling fails. No more unnoticed failures
  3. jskladan is in the process of rewriting current bodhi watcher into watching koji -pending tags
  4. Anaconda and depcheck tests were removed from master branch. There are not complete yet, we will re-add them once they are finished
  5. clumens has written a new anaconda_storage test, it's present in the clumens branch
  6. mkrizek worked heavily on posting Bodhi comments. It's almost finished, expecting additional changes this week
Next steps ...
  1. Wwoods posting blog article explaining the need for autoqa ticket#248
  2. Jskladan rewriting bodhi watcher to work with depcheck (encountering depcheck issues)

Open discussion - <Your topic here>

mdomsch - FTBFS

Owner 
mdomsch
Summary
my FTBFS report includes ~80 bugs that now BFS
I'd appreciate someone _else_ verifying the logs, then closing them
begin creating the list of FTBFS bugs that will cause packages to be blocked from F15 at Alpha
starting with those ~110 that are still open from F14, and warning people
Next steps...
Team didn't feel FTBFS issues were an appropriate fit for QA group
Bring FTBFS discussion to rel-eng or FESCO

pjones - grub 4kb bugs

Owner 
pjones
Summary
https://fedoraproject.org/wiki/Features/FourkBSectorBooting
rawhide/f15 now have support for booting from drives with 4kB sectors (as opposed to 512B like normal disks have) for UEFI only
Next steps ...
Folks installing rawhide starting from last Friday (12-03), be on the lookout for boot related failures

AdamW - Critpath test plans

Owner 
AdamW
Summary
https://fedorahosted.org/fedora-qa/ticket/154
Adam is working on the wiki structure/layout to begin creating critpath test cases and plans
Next steps ...
  1. Experiment with wiki document structure
  2. Identify how testers (and tools/bodhi) will locate tests/plans
  3. Begin drafting specific critpath tests

Action items

IRC Transcript

jlaska #startmeeting Fedora QA meeting 16:00
zodbot Meeting started Mon Dec 6 16:00:23 2010 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00
zodbot Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00
jlaska #meetingname fedora-qa 16:00
zodbot The meeting name has been set to 'fedora-qa' 16:00
jlaska #topic Gathering ... 16:00
* jsmith lurks 16:00
jlaska jsmith: lurker! 16:02
jsmith Aye... guilty as charged 16:02
* jsmith is juggling cats this morning 16:02
* mkrizek here 16:02
jlaska mkrizek howdy 16:02
adamw yo 16:02
* jlaska electronically waves to adamw 16:03
adamw isn't that illegal in most states? 16:03
jlaska and some territories 16:03
adamw =) 16:04
* Viking-Ice here 16:04
jlaska Anyone else around? I suspect kparal is lurking, Viking-Ice, robatino, wwoods, jskladan etc... 16:04
jlaska ah there we go, Viking-Ice: hiya 16:04
Viking-Ice Greetings 16:04
* kparal forgot to announce himself :) 16:04
jlaska wait another minute or so here ... and then will get started 16:04
jlaska kparal: no need, we sensed your presence :) 16:04
jlaska mkrizek: are we causing you to miss another lecture? 16:05
mkrizek btw jskladan says hi, he had to go to school, he's taking a final exam today 16:05
mkrizek jlaska: yes:) 16:05
* fenrus02 waves 16:05
jlaska fenrus02: howdy 16:06
jlaska oh no ... good luck jskladan 16:06
pjones jlaska: another item for open floor: pay attention for grub failures. I'm somewhat worried about effects of the 4kB sector patch I put in thursday. 16:06
jlaska pjones: that's coming in rawhide/F15? 16:06
pjones it's in rawhide now. 16:06
jlaska pjones: I'll bug ya for more details once we get there if that's okay 16:07
pjones alright, just ping me when you get there. 16:07
jlaska k 16:07
jlaska let's get movin' 16:07
jlaska #topic Previous meeting follow-up 16:07
jlaska #info jlaska to prep F15 test day wiki and request ideas on test@ 16:08
jlaska the wiki templates/categories have been created ... I think I got all the pages 16:08
jlaska I started tossing ideas into hte discussion page at https://fedoraproject.org/wiki/Talk:QA/Fedora_15_test_days 16:08
adamw cool 16:08
adamw i'll fill that out, actually have quite a lot of ideas lined up already 16:08
jlaska adamw: sweet ... I figured the GNOME3 ideas would expand based on your review 16:09
jlaska adamw: do we want to keep using that talk page as a "whiteboard" for ideas? 16:09
jlaska or do we want to move this stuff into TRAC tickets soon? 16:09
adamw yeah, trac tickets is good 16:10
adamw but a talk page is fine for quickly gathering the ideas 16:10
jlaska okay ... anyone want to fire off something to test@ to get some more ideas going? 16:11
* jlaska needs to add dramsey's dual-boot event to the list 16:11
jlaska alright ... next was just a check-in with nirik ... 16:11
jlaska #info Nirik to start discussion on test list related to updates ideas (see http://lists.fedoraproject.org/pipermail/test/2010-November/095756.html) 16:11
jlaska nirik: around? 16:12
nirik just barely. ;) 16:12
jlaska heh, know the feeling :) 16:12
jlaska nirik: if you had a moment, just wanted to follow-up with you on the thread you started last week 16:12
nirik ok. 16:12
jlaska Are you happy with the amount of feedback received? Are there still gaps in your opinion? 16:12
nirik I meant to do some replying/collecting of ideas from there... 16:12
nirik it seemed like good info/feedback. More is always welcome. 16:13
jlaska certainly 16:13
jlaska I think we addressed all the big items on the list ... but shout if there are any areas not detailed enough 16:13
jlaska alright ... main agenda time ... 16:14
jlaska #topic Many ways to get involved in QA ... 16:14
adamw hold up 16:14
adamw ... 16:14
adamw d'oh 16:14
jlaska adamw: waddya got? 16:14
jlaska #topic Previous meeting follow-up 16:14
adamw just wanted to note one thing: we're clearly *really* waiting on Bodhi v2.0 (Now With Added Unicorns) 16:14
adamw the non-numeric feedback is needed for all sorts of stuff 16:15
nirik yeah. Any eta on that? 16:15
adamw anything we (possibly an RH 'we' there) could do to kick it along would help 16:15
adamw nirik: no, we've asked luke a few times but never quite pinned him down to a date 16:15
nirik ok. 16:15
jlaska okay 16:16
jlaska adamw: thanks for the extra note ... I'll move on since I don't think there is much we can tackle on that subject here 16:18
adamw yeah, just wanted to highlight it 16:18
jlaska #info A lot of critical features planned for Bodhi-2.0 release, but unclear on ETA 16:18
jlaska #topic Many ways to get involved in QA ... 16:18
jlaska okay, this topic is kind of silly ... but I just wanted to point out that all of the following activites are ways for people to get involved 16:19
jlaska and I think we have plenty for everyone 16:19
jlaska we already talked about ... 16:19
jlaska #info Call for F15 Test Day ideas - https://fedoraproject.org/wiki/Talk:QA/Fedora_15_test_days 16:19
jlaska In discussion from some of the F-14 fedora-qa ticket cleanup, bruno mentioned this one ... 16:19
jlaska #info Tweak python script to provide list of UNTESTED blocker bugs - see https://fedorahosted.org/fedora-qa/ticket/89#comment:11 16:20
jlaska Two bigger efforts that will need more discussion and ideas ... 16:20
jlaska #info Requirements review for Fedora test case management - see http://fedorahosted.org/fedora-qa/ticket/152 16:21
jlaska #info Critical Path test case development - http://fedorahosted.org/fedora-qa/ticket/154 16:21
adamw yup! 16:21
jlaska and of course ... the ever-present AutoQA project has no shortage of tasks and tickets for folks to help with :) 16:21
jlaska anything I missed in the general PSA? 16:21
adamw not that i can think of 16:22
jlaska we have few bodies, and plenty of tasks 16:22
jlaska which is much more exciting than plenty of bodies and no tasks 16:22
jlaska so if anyone wants to get involved ... I think we can find something tailored to any interest level 16:23
jlaska okay okay ... I'll quit the PSA :) 16:23
jlaska kparal: you ready for the main event? 16:23
* kparal is already pre-typing the text :) 16:23
jlaska #topic AutoQA update 16:24
kparal Here we go. Updates from AutoQA for the last week: 16:24
jlaska kparal: I've got last weeks 'next steps' listed on the meeting wiki page if that helps (https://fedoraproject.org/wiki/QA/Meetings/20101206#AutoQA_Update) 16:24
kparal ah 16:24
kparal I'll come to that :) 16:24
jlaska okay 16:24
kparal 1. All autoqa config files are now copied from the server to the client when executing tests. That will allow us to use "private" config files like fas.conf (containing credentials necessary for posting comments into Bodhi). It will also allow us to use local config files (e.g. checked out from git) instead of the system-wide ones. But more about that in the future. 16:24
jlaska #info All autoqa config files are now copied from the server to the client when executing tests. 16:25
kparal 2. Watchers and autoqa script now return non-zero return code when test scheduling fails. No more unnoticed failures. 16:25
kparal 3. jskladan is in the process of rewriting current bodhi watcher into watching koji -pending tags. During the development we have found some problems with current bodhi implementation, which persuaded us that the rewrite is necessary. 16:26
jlaska kparal: ignore me ... I'm back-filling with meetbot notes 16:26
jlaska #info Watchers and autoqa script now return non-zero return code when test scheduling fails. No more unnoticed failures. 16:26
jlaska #info jskladan is in the process of rewriting current bodhi watcher into watching koji -pending tags. 16:26
kparal if anyone has any comments, feel free to post them, don't hesitate :) 16:26
kparal 4. Anaconda and depcheck tests were removed from master branch. There are not complete yet, we will re-add them once they are finished. 16:27
jlaska I have one ... 16:27
jlaska rockin'! :) 16:27
kparal jlaska: yes please? 16:27
kparal ah, thanks :) 16:27
jlaska kparal: you know me ... I save the more boring comments for the list :) 16:27
jlaska #info Anaconda and depcheck tests were removed from master branch. There are not complete yet, we will re-add them once they are finished. 16:27
kparal 5. Fedora 12 is EOL, we have removed it from the watchlist 16:27
jlaska good bye sweet F-12, you will be missed 16:28
* jlaska checks autoqa.fp.org repoinfo.conf file 16:28
kparal 6. clumens has written a new anaconda_storage test, it's present in the clumens branch. It is a very complex test, that should cover most of the installation test cases from our release validation matrix 16:28
kparal kudos to him 16:28
jlaska kparal: I just updated production to disable F-12 test scheduling 16:29
kparal jlaska: thanks 16:29
jlaska yeah, really nice work clumens! ... you can read more at -- http://www.bangmoney.org/serendipity/index.php?/archives/161-This-Week-in-Anaconda-7.html 16:29
kparal after anaconda_storage is in master, we should have anaconda installation tests executed after each anaconda build. yay! 16:29
jlaska #info clumens has written a new anaconda_storage test, it's present in the clumens branch. 16:29
jlaska kparal: and some day ... after each git commit :) 16:30
kparal almost scary :) 16:30
jlaska hehe 16:30
jlaska "Your honor, this automated world frightens and confuses me" 16:30
adamw this court rules that you suck it up, princess 16:31
kparal 7. mkrizek worked heavily on posting Bodhi comments. we have now it almost finished, but jlaska proposed some great further enhancements, so the patch will be probably adjusted and posted again this week 16:31
jlaska adamw: haha 16:31
kparal mkrizek: do you want to go into detail about his one, or should I continue? 16:31
jlaska #info mkrizek worked heavily on posting Bodhi comments. It's almost finished, expecting additional changes this week 16:32
jlaska mkrizek: is probably mad at me ... we just won't let him close that ticket! 16:32
mkrizek kparal: please do:) 16:32
mkrizek jlaska: haha 16:32
mkrizek jlaska: no, those were great comments 16:32
kparal oh, and he added architecture support into that patch (reading from last week's next steps) 16:33
jlaska nice, I missed that ... good stuff 16:33
kparal ok, I think I have covered everything. can't extract any more important information from my last week's log :) 16:34
jlaska sounds like a busy, but productive week 16:34
jlaska thanks for the updates 16:34
kparal you're welcome 16:34
jlaska any questions/comments for kparal and company before we move on? 16:34
adamw any news on depcheck? 16:35
jlaska is wwoods around? 16:35
kparal wwoods promised some blog post article. apart from that, we didn't touch depcheck in the last week 16:36
kparal we need the rewritten watcher first 16:36
adamw ok 16:37
jlaska iirc, he's also working on http://fedorahosted.org/autoqa/ticket/248 16:37
jlaska which is related to the blog post I think 16:37
jlaska basically ... we need a way to keep multiple tests from clobbering each others results 16:38
* Viking-Ice points on if reporters are supposed to be required to comment in bodhi maintainers should be required to provide test cases for component.. 16:38
jlaska Viking-Ice: I don't think we're too far off from having that discussion. imo ... that's related to the ticket/thread adamw kicked off on test@ 16:39
jlaska alright ... anything else on autoqa? 16:39
kparal nope 16:39
jlaska okay ... let's dive into open discussion 16:39
jlaska pjones: still around? 16:39
pjones yep 16:39
jlaska if so ... you can go first (so we don't lose you for lunch) 16:39
pjones okay, well, https://fedoraproject.org/wiki/Features/FourkBSectorBooting 16:40
jlaska #topic Open discussion - pjones - grub 4kB sectors bugs 16:40
jlaska #info https://fedoraproject.org/wiki/Features/FourkBSectorBooting 16:40
pjones this is support for booting from drives with 4kB sectors (as opposed to 512B like normal disks have) 16:40
pjones it's only on UEFI, but some of the code changes are in common codepaths and they're a bit hairy 16:40
pjones so I'm just asking people to keep an eye out - if you see new grub installations start and then fail to work, it may be related. 16:41
pjones I've tested it locally and it seems to work for me, but there's some risk still. 16:41
jlaska what would fail look like ... grub-install failing, or boot failures? 16:41
pjones boot failures. most likely you'd see grub start and then who knows what. 16:43
jlaska okay 16:43
jlaska would this affect anyone running rawhide today? 16:43
jlaska s/would/could/ 16:43
pjones the major changes that I'm most concerned with are in the disk buffering. 16:43
pjones on installs friday and later, yes. 16:43
jlaska okay 16:43
pjones we don't re-install grub by default if you do a package upgrade, so if you've got an older install, it probably won't effect you. 16:43
jlaska any recovery procedure if someone gets bit by this? 16:44
jlaska aaah 16:44
pjones boot a rescue image and install an older version of grub, then run grub-install. 16:44
jlaska okay 16:44
pjones and, you know, be sure and report it ;) 16:44
jlaska #info rawhide/f15 now have support for booting from drives with 4kB sectors (as opposed to 512B like normal disks have) 16:44
jlaska pjones: details details ;) 16:44
pjones #info only on UEFI 16:44
jlaska thx 16:45
pjones the feature hasn't actually been approved yet, but, well... 16:45
fenrus02 are any drives shipping with 4k yet? 16:45
pjones on the market? no. 16:45
pjones right now they're NDA-only. 16:45
* fenrus02 nods 16:45
jlaska pjones: thanks for the heads up 16:46
pjones which limits our testing a bit. But it works on my box ;) 16:46
jlaska pjones: when will drives start landing with this support? 16:46
pjones np. 16:46
pjones no idea. 16:46
jlaska heh, surprise! 16:46
pjones I think that's actually still classified as an "if" rather than a "when", actually. 16:46
jlaska okay 16:46
jlaska alrighty ... next topic then 16:47
jlaska #topic Open discussion - mdomsch - FTBFS 16:47
jlaska #link http://lists.fedoraproject.org/pipermail/devel/2010-December/146683.html 16:47
jlaska #link http://lists.fedoraproject.org/pipermail/devel/2010-December/146682.html 16:47
jlaska mdomsch2: still around? 16:47
jlaska mdomsch asked, "my FTBFS report includes ~80 bugs that now BFS, I'd appreciate someone _else_ verifying the logs, then closing them" 16:47
mdomsch2 not really 16:47
jlaska additionally, "begin creating the list of FTBFS bugs that will cause packages to be blocked from F15 at Alpha. starting with those ~110 that are still open from F14, and warning people" 16:48
jlaska mdomsch2: okay 16:48
jlaska perhaps nirik might know as well 16:48
jlaska I'm not extremely familiar with the fallout of FTBFS packages? 16:49
nirik whats the question? ;) 16:49
jlaska these are existing packages, that no longer build in rawhide/F15 and will need updates? 16:49
adamw i'm not sure about the 'will cause packages to be blocked' thing 16:49
adamw do we block packages for ftbfs? 16:49
dgilmore adamw: nope 16:49
jlaska this doesn't strike me as a classic QA task ... but I might be forgetting history here 16:50
nirik we didn't last cycle... but there was plans to. 16:51
nirik http://fedoraproject.org/wiki/FTBFS 16:51
mdomsch2 well, we did purge ftbfs pkgs once 16:51
nirik see last point. 16:51
mdomsch2 we should at alpha compose 16:52
adamw yeah, this kinda feels more like devel stuff 16:52
nirik well, or releng... 16:52
mdomsch2 or fesco? 16:52
adamw or anyone but us! 16:52
adamw give it to the design team, they'll love it 16:52
mdomsch2 i'd love it if it wzsn't just me driving it 16:53
jlaska yeah, I don't mean to push off tasks, but I agree ... this feels like a process rel-eng would manage at least? 16:53
jlaska dgilmore ? 16:53
mdomsch2 it's a meta qa task imho 16:53
adamw i don't think we ever really expanded QA's remit to 'does it build' 16:54
mdomsch2 is the distro self-hosting? 16:54
dgilmore jlaska: we dont block packages that ftbfs 16:54
nirik dgilmore: we have, we just didn't this last cycle. ;) 16:55
mdomsch2 I can take it up elsewhere if qa isn't interested, no problem 16:55
dgilmore nirik: hrrm, i guess i missed that 16:55
adamw just feels like it's more likely to result in fixes if it goes under releng/devel 16:55
nirik mdomsch2: possibly fesco or rel-eng should discuss it? 16:55
mdomsch2 k 16:55
adamw sure, you can argue the 'find out what builds and what doesn't' bit is QA, but now you have a script that bit is quite trivial: run the script 16:55
jlaska mdomsch2: it's not so much that we aren't interested ... but more that deciding whether we are self-hosting seems to be a multi-group/fesco thing 16:55
adamw the hard bit is 'make it build', and that's work for devel 16:55
adamw to me it feels like the notifications of dependency issues, for instance 16:56
adamw they go to -devel and aren't part of qa stuff 16:56
jlaska good comparison 16:56
jlaska mdomsch2: btw ... is generation of these reports a manual process now? 16:56
abadger1999 Should qa be in charge of the "verify that it now builds before closing bug" portion? 16:57
jlaska hmmm 16:57
adamw that feels like needless make-work 16:58
jlaska that seems unnecesary ... but this might be my lack of understanding around FTBFS 16:58
adamw it's much more efficient for the developer to just do it when the build succeeds 16:58
adamw the developer knows what the bug number is, knows when they did the build, and knows what the build number is =) 16:58
jlaska yeah, the risk of allowing the maintainer to manage this process seems low 16:58
adamw i'm pretty sure the devs aren't going to start lying about it. and if they do, all that would happen would be a new bug got filed next time the script ran. 16:59
jlaska good discussion, good ideas 16:59
jlaska alrighty ... anything else on this topic, or other topics? 17:00
jlaska #topic Open discussion - <your topic here> 17:00
adamw i have a quick topic 17:00
jlaska adamw: take it away 17:00
jlaska #chair adamw 17:01
zodbot Current chairs: adamw jlaska 17:01
adamw critpath test plans! 17:01
jlaska #topic Open discussion - Critpath test plans 17:01
adamw jlaska mentioned it earlier, but just to trumpet it 17:01
adamw https://fedorahosted.org/fedora-qa/ticket/154 17:01
adamw we need test plans for critical path packages! I'll finish the groundwork today 17:01
adamw I'm probably going to set up a proposed standard naming scheme for the plans, start filing tickets per component, and write an example test case 17:02
adamw then it'd be great if others could help contribute test cases 17:02
adamw the idea being 'this case / set of cases makes sure the critical path-related functionality of this package is working' 17:03
jlaska adamw: sweet, I think the biggest road block for contributions will be settling in on a standard content convention 17:03
adamw i don't think the *content* of the test cases needs to be standardized 17:03
adamw in fact it'd probably be a bad idea 17:03
jlaska errmm no 17:03
adamw ah =) 17:03
jlaska I see what you're saying 17:03
adamw maybe i misunderstood you 17:03
jlaska but let's keep things to the 'QA/Test_Case' template for now 17:03
adamw well, i was thinking of something after that 17:04
adamw a further prefix 17:04
jlaska and by convention ... I meant what you refered to earlier about the wiki structure (categories, naming convention etc...) 17:04
adamw maybe just the package .src.rpm name 17:04
adamw or packagename_critpath_(whatever) 17:04
jlaska great phase of the project to play around with different options 17:05
jlaska the choice is important as it helps testers find content, and allows other tools to link to appropriate content? 17:05
adamw so we'd have QA/Test_Case_NetworkManager_critpath_connection or something 17:05
adamw yes, the second is the important bit: bodhi integration is what i'm thinking about 17:05
adamw as i mentioned it'd be great if bodhi and f-e-k can automatically integrate with the set of test cases for the update in question 17:06
jlaska spend extra time on the wiki layout/organization step 17:06
adamw i guess the other thing we can do is have a (sub) category for each component 17:06
jlaska that'll definitely pay off 17:06
jlaska adamw: cool, you want this on the agenda for next week then? 17:07
adamw sure, 17:07
adamw why not 17:07
jlaska adamw: I'll bug you by end of week 17:07
jlaska this will be a nice foot in the door to building our test documentation ... thanks for takin this on 17:08
adamw npnp 17:08
jlaska alrighty ... let's close out on 30 seconds 17:08
jlaska 10 seconds ... 17:09
jlaska alright, thanks all! 17:09
jlaska #endmeeting 17:09

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!