QA/Meetings/20130218

From FedoraProject

Jump to: navigation, search

Contents

Attendees

  • adamw (120)
  • Viking-Ice (47)
  • nirik (27)
  • cwickert (25)
  • tflink (21)
  • kparal (12)
  • satellit (6)
  • zodbot (3)
  • jskladan (1)
  • mkrizek (1)
  • robatino (1)
  • pschindl (1)

Agenda

  • Automatic blocker proposal
  • Blocker review process
  • Open floor

Automatic blocker proposal

  • The proposal was generally approved of by the group
  • It was agreed to add Andre's suggestions and more strongly emphasize other bugs cannot be made automatic blockers

Blocker review process

  • Limiting meetings to 3 hours and separating blocker review from QA meetings on Mondays seemed to work well during F18 cycle
  • viking-ice proposed using a dedicated channel for blocker review meetings instead of -qa or -bugzappers
    • tflink and kparal had concerns about having another channel to monitor, but adamw and viking pointed out this would mean not needing to use -bugzappers any more

Open floor

  • viking-ice wondered whether the nightly images were worth the effort of building: nirik said that building them is not much effort, and adamw and others said they were aware of some testing being done on them
  • viking-ice suggested we have a basic smoke test matrix that all spins must go through at least once before being promoted as part of a final release
    • cwickert said he had made a similar proposal before and supported the idea

Action items

  • adamw to write a second draft (of the automatic blocker proposal) with andre's proposed changes and stronger explanation not to put 'grey area' bugs in the automatic blocker list
  • adamw to draft up changes to the blocker bug meeting SOP for 3-hour hard limit, no-reviews-during-qa-meetings, and a dedicated channel for meetings, send to list for further discussion
  • viking-ice to discuss the 'smoke test for spins' idea further with nirik and cwickert

IRC Log

adamw #startmeeting Fedora QA meeting 16:00
zodbot Meeting started Mon Feb 18 16:00:05 2013 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00
zodbot Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00
adamw #meetingname fedora-qa 16:00
zodbot The meeting name has been set to 'fedora-qa' 16:00
adamw #topic roll call 16:00
adamw morning folks, who's around? 16:00
* kparal waves 16:00
* satellit listening 16:00
* mkrizek is here 16:00
* nirik is lurking 16:01
* pschindl is here 16:01
* tflink is here 16:02
adamw hey, the gang's all here, now- OH NO WATCH OUT FOR THAT METEOR DRIVEN BY A RAPTOR 16:02
kparal jskladan will survive and free us from the raptor dictatorship 16:03
adamw wow, tough crowd. 16:03
adamw heh 16:03
* jskladan lurks 16:06
* adamw waves from under meteor 16:06
adamw hum, we don't seem to have a viking-ice yet 16:06
adamw tflink: do you know what it was he wanted to discuss about the review process? 16:07
kparal oh no, we're one viking short 16:07
adamw if no-one knows what it was he wanted to talk about, we'll skip that item 16:09
tflink adamw: wasn't it on the agenda for last week 16:09
tflink nvm 16:10
tflink oh, it was about the changes we kept making in F18 16:10
adamw oh, the "QA:TestCase" topic from 0128? 16:10
tflink keeping a static IRC channel, capping meetings @ 3hrs etc. 16:10
adamw oh, i see. 16:10
adamw well, let's do the other topic first 16:11
adamw #topic Automatic blocker proposal 16:11
adamw seems like most of the feedback on the 'automatic blocker' idea is +ve, i'll adjust it to incorporate andre's suggestions, any other thoughts? 16:11
adamw #info the proposal is https://lists.fedoraproject.org/pipermail/test/2013-February/113840.html 16:12
tflink it seems like a good idea to me 16:12
tflink but it will increase the average time that we spend on bugs in meetings though :) 16:12
kparal tflink: will it? 16:12
tflink since we won't have the really easy ones bringing the average down 16:12
kparal ah 16:12
kparal the _average_ time 16:12
kparal yes, bad for statistics! nack! 16:12
kparal  :) 16:12
adamw heh 16:13
adamw lies, damn lies, and tflink statistics 16:13
robatino will there be automatic freeze exceptions as well (for oversized non-blocking desktops, for example)? 16:13
adamw robatino: i didn't reply to that mail yet, but it seems reasonable 16:13
adamw i'll try and come up with a new draft soon; i might emphasize the rules a bit harder too 16:13
adamw so no-one can claim they misread it and just start slapping acceptedblocker on everything they propose 16:14
tflink yeah, hopefully this won't be abused 16:15
tflink but we won't know until we try 16:15
adamw ah, the viking's here 16:16
adamw Viking-Ice: we're on the 'automatic blocker' proposal - any further thoughts on that? 16:16
Viking-Ice nope I agree to it fully 16:16
Viking-Ice ( I needlessly worried a bit about that gray area )( 16:17
adamw cool 16:18
adamw ok, i'll send out a second draft with andre's suggestions soon then 16:18
adamw #info group generally supports the automatic blocker proposal 16:18
adamw #action adamw to write a second draft with andre's proposed changes and stronger explanation not to put 'grey area' bugs in the automatic blocker list 16:18
adamw #topic Blocker review process 16:19
adamw Viking-Ice: we held this one in case you showed up - so, you said you wanted some discussion about this? 16:19
Viking-Ice nothing comes to mind at the moment 16:21
Viking-Ice so I got nothing new to add atleast 16:21
* adamw checks log 16:22
adamw Viking-Ice well what did we learn about the blocker bug meetings 16:22
adamw Viking-Ice well we should set a fixed channel and keep with the 3hour max limit 16:22
adamw tflink #info discussion around the blocker review process for F19 would be wise before we get into testing 16:22
adamw okay, that's where we were coming from. 16:22
adamw so i guess this is about whether we want to formalize any of the f18 changes to the blocker bug meeting process 16:23
Viking-Ice I think the 3 hour limit turn out working well but still I doubt ( or let's say I hope ) that we wont be experiencing that again this release cycle 16:24
tflink we can hope :) 16:24
adamw we all hope so :) 16:24
adamw but it does seem like a reasonable rule indeed 16:24
tflink but I suspect that it's going to keep happening every once in a while until/if we redo the process 16:24
tflink but that's not happening for F19 16:25
Viking-Ice and perhaps we should introduce new channel dedicated just for this ( not qa as some people wanted and not bugzappers and not meeting ) 16:25
Viking-Ice I also think it's better not to do blocker bug meetings in the midst of qa meetings or atleast I think it's better to just end the qa meeting and move to another channel 16:26
adamw we did that once in the last cycle and it worked out fine 16:26
adamw so it's a decent idea 16:26
tflink yeah, I don't have any objections 16:27
adamw it does get a bit messy having to look in qa meeting logs for blocker review 16:27
adamw i can draft up a few changes to the sop 16:27
tflink not sure about the dedicated channel, though unless we re-purpose #fedora-bugzappers 16:27
adamw eh 16:27
adamw i don't mind it 16:27
Viking-Ice the reason I personally favor moving/using qa channel is likely hood of more participation 16:27
adamw not like channels cost anything 16:27
Viking-Ice tflink, we cant kill bugzappers if we continue to use it 16:28
tflink it's just one more channel to join and keep an eye on :) 16:28
nirik channels actually do cost. ;) 16:28
adamw i can see viking's argument that using -bugzappers is kinda weird 16:28
adamw the only reason to use -bugzappers any more is for these meetings though 16:28
nirik they cost in attention of people... 16:28
adamw so the net cost of a new channel is 0, as if we used one, everyone could quit -bugzappers... 16:28
tflink that'd work for me 16:28
tflink I don't much care about what the channel is named 16:28
tflink but using a dedicated channel does open some interesting possibilities with irc bots in the future 16:29
adamw Viking-Ice: i think we convinced him ;) 16:29
Viking-Ice everyone is familiar and usually on the qa channel ( devs/qa community members ) but we might be interrupted like happened that one time if we use it in the midst of the meeting 16:30
adamw okay, so how about this, i'll draft sop changes for all the above ideas and we can kick it around further on list 16:30
adamw yeah, that's the problem with using -qa 16:30
adamw it's a pretty active channel 16:30
Viking-Ice I assume we want as much activity on that channel 16:31
Viking-Ice ( which usually means more vibrant and active community ) 16:32
* nirik is happy with another channel as long as we kill bugzappers. net 0 is good. 16:32
adamw okay. 16:32
adamw Viking-Ice: sure, we want -qa to be active, but as you said, it gets awkward if we're having a two-hour blocker meeting and someone shows up wanting to chat about something else. 16:32
tflink yeah, agreed that #fedora-qa is not the right place for review meetings 16:33
Viking-Ice just throw it on the test list new channel any suggestion for the name of that channel and or use the qa channel and see how the community reacts/wants it 16:33
adamw sounds good. 16:33
tflink it sounded like a good idea when it was first proposed but in reality, it caused more problems than it solved :-/ 16:34
adamw #action adamw to draft up changes to the blocker bug meeting SOP for 3-hour hard limit, no-reviews-during-qa-meetings, and a dedicated channel for meetings, send to list for further discussion 16:34
adamw okay then 16:36
adamw looks like that's all we had on the agenda, so... 16:36
adamw #topic open floor 16:36
satellit anymore koji builds to test f19? 16:36
adamw koji builds? 16:37
satellit lives to test 16:37
Viking-Ice I've been wondering a bit about that do we really need iso files ? 16:37
Viking-Ice ( other then alpha beta final ) 16:37
Viking-Ice as in nightly's 16:37
nirik satellit: I have been holding off doing them while the mass rebuild is running. Should resume tomorrow or so. 16:38
adamw they're useful, sure. 16:38
Viking-Ice aren't we usually using them only to test anaconda? 16:38
satellit I rely on the .iso's for soas 16:38
adamw in f18 cycle we didn't use them a lot as we were making TCs almost constantly 16:38
adamw but in previous cycles they've gotten a decent amount of use. not that weird to ask someone to check something with a nightly. 16:38
kparal also tflink's composes lowered our usage of nightlies 16:39
Viking-Ice do we have download stats on the iso's 16:39
adamw not sure koji tracks that...nirik? 16:40
adamw kparal: we're aiming to do fewer smoke builds for f19, to save tflink all the work. 16:41
nirik I don't know that it does off hand... 16:41
nirik there's probibly http logs. 16:41
* nirik could look if you like. 16:41
Viking-Ice I've briefly been touching/pondering the idea if we somehow can use Colin Walters OStree to our advantage ( https://live.gnome.org/OSTree ) 16:41
adamw i remember reading his blog post on it and thinking 'hmm, that's interesting', but i didn't really have any concrete ideas 16:42
nirik so, httpd logs are kept for iso downloads. What info from there would you find useful? 16:43
adamw i think viking was curious about how much the nightlies are downloaded? 16:43
nirik 4569 downloads in 2012-12 16:44
Viking-Ice adamw, yeah that's where I'm at came across it looks interesting wondering if we can take some kind of advantage of it but nothing concrete yet 16:44
Viking-Ice nirik, each release? 16:44
nirik 5117 in 2013-01 16:44
Viking-Ice or total 16:44
nirik total 16:44
nirik any ".iso " download 16:44
nirik nightlys are only kept for a week or so tho. 16:45
Viking-Ice I'm just trying to asses the benefit of using it 16:45
Viking-Ice vs overhead 16:45
adamw the overhead's pretty tiny 16:45
adamw i think it's just nirik firing a script 16:45
nirik yep. 16:46
adamw if it doesn't build, we don't try and fix it 16:46
nirik and we keep wanting to automate it. 16:46
Viking-Ice makes sense 16:46
nirik it's useful also for spins folks to test if they ever do 16:46
satellit +1 16:46
Viking-Ice technically gnome users should be testing the gnome spin as well while we try to focus our energy on the core function 16:47
Viking-Ice but yeah 16:47
Viking-Ice how much testing did other then the *DE spin get 16:48
Viking-Ice I think those might be getting little to no testing even from their maintainers 16:49
nirik I don't think there's any reasonable way to quantify that. ;) 16:49
adamw in f18 not a huge lot, for f15->f17 i tried to get decent amount of testing for the non-blocking spins 16:49
* nirik did in fact test the Xfce spin a number of times. 16:49
adamw we at least made sure the whole desktop matrix was done once or twice at each milestone 16:49
adamw for xfce and lxde 16:49
adamw satellit tests sugar quite a lot 16:50
Viking-Ice I'm not worried about the *DE spins ( and sugar ;) ) they all have active communities it's the other ones that concern me 16:50
satellit I also do VirtualBox installs from spins to test yum installs of other DE's with sugar 16:50
adamw outside of the desktops and sugar, hell if i know. 16:51
adamw you may well be right that they don't get much of a look. 16:51
Viking-Ice I'm wondering if we should not come up with a test matrix for those that the spin maintainers have to walk through and "pass" before release 16:51
* nirik nods. Suggested as much to the spins list a while back. 16:51
adamw i don't mind the idea in theory, as it does kinda suck when we ship stuff that's utterly borked 16:51
adamw even if it's a spin we explicitly don't support 16:52
adamw maybe you two could get together and re-propose it to spins? 16:52
Viking-Ice what's releng take on something like that 16:52
Viking-Ice ( anything we might handout at various events needs to be thoroughly tested ) 16:53
nirik the spins setup is disfunctional, but attempts to fix it haven't met with anything concrete. 16:53
nirik cwickert would be the one to involve in those discussions 16:53
cwickert  ? 16:54
nirik cwickert: spins process... didn't go so well last cycle. ;( 16:54
cwickert yes, I know 16:54
Viking-Ice cwickert, to bring you up to speed qa/releng requesting test matrix spin has to pass before being released 16:55
cwickert but I'm afraid it will become worse when we kill the spins 16:55
adamw i don't think we hand out anything but the multi-install and multi-desktop 16:55
adamw and the regular install / desktop of course 16:55
cwickert Viking-Ice: that means what exactly? 16:55
nirik I was thinking a 2 person checkoff of a test matrix for each spin we want to promote on spins.fedoraproject.org. The rest can exist, just in a corner of alt. 16:56
Viking-Ice cwickert, test matrix spin maintainers have to walk through which ensures atleast no surprises 16:56
Viking-Ice for their spins 16:56
adamw this is the proposal 16:56
cwickert Viking-Ice: argh 16:57
Viking-Ice cwickert, the *de spins are not much worries since those have active community's it's the other spins 16:57
cwickert Viking-Ice: please consider me as an idiot who doesn't have a clue what a "test matrix spin maintainer" is 16:57
adamw cwickert: there should've been some punctuation or grammar in there :) 16:57
Viking-Ice uhum yes 16:58
cwickert I know what a test is, I know what a test matrix is, I know what a maintainer is 16:58
adamw the idea is that there would be *a* test matrix (basically just a test plan) that spin maintainers have to run through to have their spin 'approved' or whatever for a release, just a very basic 'does it boot?' smoke test 16:58
nirik I think we are talking about "it boots, selinux is enforcing and works, it lets you login, etc" 16:58
cwickert but who is supposed to maintain the test matrix for a spin? 16:58
adamw cwickert: it'd be a generic one i think 16:58
cwickert can there be specific tests for a spin? 16:59
cwickert who is to maintain them and so on 16:59
cwickert there is tons of questions 16:59
adamw i think this idea would just be a very basic generic 'smoke test' 16:59
adamw spin-specific tests are possible but would be a different thing 16:59
Viking-Ice qa would maintain the matrix I suppose we already have criteria for "core" the rest is just packages on top of that 16:59
adamw we actually already have one such matrix for the security lab spin (though no-one ever runs it) 16:59
cwickert ok, I'm sorry, I need to stop here, FAMSCo meeting 16:59
cwickert but we DO need to talk about this 16:59
adamw anyway, it seems like a decent idea 16:59
* nirik nods. 17:00
adamw #action viking-ice to discuss the 'smoke test for spins' idea further with nirik and cwickert 17:00
cwickert I know the spins went badly 17:00
nirik yeah, I don't think anyone disagrees... just how we improve them. ;) 17:00
cwickert but on the other hand I am very frustrated about getting little or no feedback from QA about my requests 17:00
adamw sorry, which requests? 17:00
Viking-Ice yeah I missed those to 17:01
cwickert adamw: changelogs in the announcements for the differenc milestones, better browsability in the wiki, meaningful renaming of the tracker bugs 17:01
adamw oh, those 17:01
cwickert are you coming to devconf? 17:02
adamw 1) i talked to andre about that one and we edited the text of the announcements somewhat to make it clearer that the 'changelog' is in the trac ticket 17:02
adamw 2) unfortunately didn't get to that one yet 17:02
adamw 3) we did that 17:02
cwickert I'd appreciate if we can discuss some things 17:02
* Viking-Ice still lost... 17:02
adamw cwickert: nope 17:02
Viking-Ice I will be there kparal as well 17:03
adamw Viking-Ice: these are requests from some time back, i don't recall exactly what the forum was but i recall the discussion now 17:03
adamw cwickert: right, you can talk to viking and kparal there (probably also jskladan) 17:03
Viking-Ice perhaps this should end up in our trac instance 17:03
adamw cwickert: did you miss the tracker bug renaming thing? cos that was a whole thing a few weeks back. 17:03
kparal in a pub :) 17:03
adamw i think it may well be there 17:03
Viking-Ice kparal, with rotten shark bits ;) 17:03
adamw i'm pretty sure i filed tickets at the time 17:03
kparal Viking-Ice: that's not really a czech speciality 17:04
adamw https://fedorahosted.org/fedora-qa/ticket/273 is the ticket for the 'browsability' thing 17:04
adamw https://fedorahosted.org/fedora-qa/ticket/272 is for the 'changelog' thing...it's not that we didn't give you any feedback, really, but andre didn't entirely agree with the proposal... 17:05
Viking-Ice 307 for some of it? 17:05
adamw that wasn't part of cwickert's request, no. but i do need to finish that up. sigh 17:05
adamw so much stuff to do 17:05
adamw anyhoo, we're a bit over time 17:06
Viking-Ice kparal, I will be bringing a box of bits for people to try as requested ;) 17:06
Viking-Ice yup 17:06
adamw so let's wrap up 17:06
cwickert adamw: 3) we did that? 17:06
adamw cwickert: the tracker bug renaming,. 17:06
cwickert adamw: I don't think so 17:06
cwickert what are the names now? 17:06
adamw cwickert: https://lists.fedoraproject.org/pipermail/test/2013-January/113405.html 17:07
kparal Viking-Ice: let's hope it's not an attempt to wipe out Brno's Red Hat office :-) 17:07
cwickert I am searching the wiki for 5 minutes now for the NTH have bugs :( 17:07
adamw cwickert: they're called FreezeException now 17:07
adamw cwickert: and they're always listed at https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Trackers 17:07
adamw which is linked from https://fedoraproject.org/wiki/Blocker_Bug_FAQ and https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process 17:08
* adamw sets fuse 17:09
* cwickert needs to bail out for the FAmSCo meeting 17:10
adamw cwickert: let us know what you think about the new names 17:11
cwickert adamw: I made a proposal, so I probably prefer what I proposed, right? 17:12
adamw well i'd *hope* not everyone thinks that way :) 17:12
adamw i had a proposal too, and so did tflink, but we both prefer the final scheme 17:12
cwickert I mean, the new names are better than the old ones, but still I consider mine better :P 17:12
adamw anyhoo 17:12
adamw time to end this nightmare! 17:12
adamw thanks for coming folks 17:12
adamw #endmeeting 17:12

Generated by irclog2html.py 2.11.0 by Marius Gedminas - find it at mg.pov.lt!