| adamw
|
#startmeeting Fedora QA meeting
|
16:00
|
| zodbot
|
Meeting started Mon Feb 18 16:00:05 2013 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot.
|
16:00
|
| zodbot
|
Useful Commands: #action #agreed #halp #info #idea #link #topic.
|
16:00
|
| adamw
|
#meetingname fedora-qa
|
16:00
|
| zodbot
|
The meeting name has been set to 'fedora-qa'
|
16:00
|
| adamw
|
#topic roll call
|
16:00
|
| adamw
|
morning folks, who's around?
|
16:00
|
| * kparal waves
|
16:00
|
| * satellit listening
|
16:00
|
| * mkrizek is here
|
16:00
|
| * nirik is lurking
|
16:01
|
| * pschindl is here
|
16:01
|
| * tflink is here
|
16:02
|
| adamw
|
hey, the gang's all here, now- OH NO WATCH OUT FOR THAT METEOR DRIVEN BY A RAPTOR
|
16:02
|
| kparal
|
jskladan will survive and free us from the raptor dictatorship
|
16:03
|
| adamw
|
wow, tough crowd.
|
16:03
|
| adamw
|
heh
|
16:03
|
| * jskladan lurks
|
16:06
|
| * adamw waves from under meteor
|
16:06
|
| adamw
|
hum, we don't seem to have a viking-ice yet
|
16:06
|
| adamw
|
tflink: do you know what it was he wanted to discuss about the review process?
|
16:07
|
| kparal
|
oh no, we're one viking short
|
16:07
|
| adamw
|
if no-one knows what it was he wanted to talk about, we'll skip that item
|
16:09
|
| tflink
|
adamw: wasn't it on the agenda for last week
|
16:09
|
| tflink
|
nvm
|
16:10
|
| tflink
|
oh, it was about the changes we kept making in F18
|
16:10
|
| adamw
|
oh, the "QA:TestCase" topic from 0128?
|
16:10
|
| tflink
|
keeping a static IRC channel, capping meetings @ 3hrs etc.
|
16:10
|
| adamw
|
oh, i see.
|
16:10
|
| adamw
|
well, let's do the other topic first
|
16:11
|
| adamw
|
#topic Automatic blocker proposal
|
16:11
|
| adamw
|
seems like most of the feedback on the 'automatic blocker' idea is +ve, i'll adjust it to incorporate andre's suggestions, any other thoughts?
|
16:11
|
| adamw
|
#info the proposal is https://lists.fedoraproject.org/pipermail/test/2013-February/113840.html
|
16:12
|
| tflink
|
it seems like a good idea to me
|
16:12
|
| tflink
|
but it will increase the average time that we spend on bugs in meetings though :)
|
16:12
|
| kparal
|
tflink: will it?
|
16:12
|
| tflink
|
since we won't have the really easy ones bringing the average down
|
16:12
|
| kparal
|
ah
|
16:12
|
| kparal
|
the _average_ time
|
16:12
|
| kparal
|
yes, bad for statistics! nack!
|
16:12
|
| kparal
|
:)
|
16:12
|
| adamw
|
heh
|
16:13
|
| adamw
|
lies, damn lies, and tflink statistics
|
16:13
|
| robatino
|
will there be automatic freeze exceptions as well (for oversized non-blocking desktops, for example)?
|
16:13
|
| adamw
|
robatino: i didn't reply to that mail yet, but it seems reasonable
|
16:13
|
| adamw
|
i'll try and come up with a new draft soon; i might emphasize the rules a bit harder too
|
16:13
|
| adamw
|
so no-one can claim they misread it and just start slapping acceptedblocker on everything they propose
|
16:14
|
| tflink
|
yeah, hopefully this won't be abused
|
16:15
|
| tflink
|
but we won't know until we try
|
16:15
|
| adamw
|
ah, the viking's here
|
16:16
|
| adamw
|
Viking-Ice: we're on the 'automatic blocker' proposal - any further thoughts on that?
|
16:16
|
| Viking-Ice
|
nope I agree to it fully
|
16:16
|
| Viking-Ice
|
( I needlessly worried a bit about that gray area )(
|
16:17
|
| adamw
|
cool
|
16:18
|
| adamw
|
ok, i'll send out a second draft with andre's suggestions soon then
|
16:18
|
| adamw
|
#info group generally supports the automatic blocker proposal
|
16:18
|
| adamw
|
#action adamw to write a second draft with andre's proposed changes and stronger explanation not to put 'grey area' bugs in the automatic blocker list
|
16:18
|
| adamw
|
#topic Blocker review process
|
16:19
|
| adamw
|
Viking-Ice: we held this one in case you showed up - so, you said you wanted some discussion about this?
|
16:19
|
| Viking-Ice
|
nothing comes to mind at the moment
|
16:21
|
| Viking-Ice
|
so I got nothing new to add atleast
|
16:21
|
| * adamw checks log
|
16:22
|
| adamw
|
Viking-Ice well what did we learn about the blocker bug meetings
|
16:22
|
| adamw
|
Viking-Ice well we should set a fixed channel and keep with the 3hour max limit
|
16:22
|
| adamw
|
tflink #info discussion around the blocker review process for F19 would be wise before we get into testing
|
16:22
|
| adamw
|
okay, that's where we were coming from.
|
16:22
|
| adamw
|
so i guess this is about whether we want to formalize any of the f18 changes to the blocker bug meeting process
|
16:23
|
| Viking-Ice
|
I think the 3 hour limit turn out working well but still I doubt ( or let's say I hope ) that we wont be experiencing that again this release cycle
|
16:24
|
| tflink
|
we can hope :)
|
16:24
|
| adamw
|
we all hope so :)
|
16:24
|
| adamw
|
but it does seem like a reasonable rule indeed
|
16:24
|
| tflink
|
but I suspect that it's going to keep happening every once in a while until/if we redo the process
|
16:24
|
| tflink
|
but that's not happening for F19
|
16:25
|
| Viking-Ice
|
and perhaps we should introduce new channel dedicated just for this ( not qa as some people wanted and not bugzappers and not meeting )
|
16:25
|
| Viking-Ice
|
I also think it's better not to do blocker bug meetings in the midst of qa meetings or atleast I think it's better to just end the qa meeting and move to another channel
|
16:26
|
| adamw
|
we did that once in the last cycle and it worked out fine
|
16:26
|
| adamw
|
so it's a decent idea
|
16:26
|
| tflink
|
yeah, I don't have any objections
|
16:27
|
| adamw
|
it does get a bit messy having to look in qa meeting logs for blocker review
|
16:27
|
| adamw
|
i can draft up a few changes to the sop
|
16:27
|
| tflink
|
not sure about the dedicated channel, though unless we re-purpose #fedora-bugzappers
|
16:27
|
| adamw
|
eh
|
16:27
|
| adamw
|
i don't mind it
|
16:27
|
| Viking-Ice
|
the reason I personally favor moving/using qa channel is likely hood of more participation
|
16:27
|
| adamw
|
not like channels cost anything
|
16:27
|
| Viking-Ice
|
tflink, we cant kill bugzappers if we continue to use it
|
16:28
|
| tflink
|
it's just one more channel to join and keep an eye on :)
|
16:28
|
| nirik
|
channels actually do cost. ;)
|
16:28
|
| adamw
|
i can see viking's argument that using -bugzappers is kinda weird
|
16:28
|
| adamw
|
the only reason to use -bugzappers any more is for these meetings though
|
16:28
|
| nirik
|
they cost in attention of people...
|
16:28
|
| adamw
|
so the net cost of a new channel is 0, as if we used one, everyone could quit -bugzappers...
|
16:28
|
| tflink
|
that'd work for me
|
16:28
|
| tflink
|
I don't much care about what the channel is named
|
16:28
|
| tflink
|
but using a dedicated channel does open some interesting possibilities with irc bots in the future
|
16:29
|
| adamw
|
Viking-Ice: i think we convinced him ;)
|
16:29
|
| Viking-Ice
|
everyone is familiar and usually on the qa channel ( devs/qa community members ) but we might be interrupted like happened that one time if we use it in the midst of the meeting
|
16:30
|
| adamw
|
okay, so how about this, i'll draft sop changes for all the above ideas and we can kick it around further on list
|
16:30
|
| adamw
|
yeah, that's the problem with using -qa
|
16:30
|
| adamw
|
it's a pretty active channel
|
16:30
|
| Viking-Ice
|
I assume we want as much activity on that channel
|
16:31
|
| Viking-Ice
|
( which usually means more vibrant and active community )
|
16:32
|
| * nirik is happy with another channel as long as we kill bugzappers. net 0 is good.
|
16:32
|
| adamw
|
okay.
|
16:32
|
| adamw
|
Viking-Ice: sure, we want -qa to be active, but as you said, it gets awkward if we're having a two-hour blocker meeting and someone shows up wanting to chat about something else.
|
16:32
|
| tflink
|
yeah, agreed that #fedora-qa is not the right place for review meetings
|
16:33
|
| Viking-Ice
|
just throw it on the test list new channel any suggestion for the name of that channel and or use the qa channel and see how the community reacts/wants it
|
16:33
|
| adamw
|
sounds good.
|
16:33
|
| tflink
|
it sounded like a good idea when it was first proposed but in reality, it caused more problems than it solved :-/
|
16:34
|
| adamw
|
#action adamw to draft up changes to the blocker bug meeting SOP for 3-hour hard limit, no-reviews-during-qa-meetings, and a dedicated channel for meetings, send to list for further discussion
|
16:34
|
| adamw
|
okay then
|
16:36
|
| adamw
|
looks like that's all we had on the agenda, so...
|
16:36
|
| adamw
|
#topic open floor
|
16:36
|
| satellit
|
anymore koji builds to test f19?
|
16:36
|
| adamw
|
koji builds?
|
16:37
|
| satellit
|
lives to test
|
16:37
|
| Viking-Ice
|
I've been wondering a bit about that do we really need iso files ?
|
16:37
|
| Viking-Ice
|
( other then alpha beta final )
|
16:37
|
| Viking-Ice
|
as in nightly's
|
16:37
|
| nirik
|
satellit: I have been holding off doing them while the mass rebuild is running. Should resume tomorrow or so.
|
16:38
|
| adamw
|
they're useful, sure.
|
16:38
|
| Viking-Ice
|
aren't we usually using them only to test anaconda?
|
16:38
|
| satellit
|
I rely on the .iso's for soas
|
16:38
|
| adamw
|
in f18 cycle we didn't use them a lot as we were making TCs almost constantly
|
16:38
|
| adamw
|
but in previous cycles they've gotten a decent amount of use. not that weird to ask someone to check something with a nightly.
|
16:38
|
| kparal
|
also tflink's composes lowered our usage of nightlies
|
16:39
|
| Viking-Ice
|
do we have download stats on the iso's
|
16:39
|
| adamw
|
not sure koji tracks that...nirik?
|
16:40
|
| adamw
|
kparal: we're aiming to do fewer smoke builds for f19, to save tflink all the work.
|
16:41
|
| nirik
|
I don't know that it does off hand...
|
16:41
|
| nirik
|
there's probibly http logs.
|
16:41
|
| * nirik could look if you like.
|
16:41
|
| Viking-Ice
|
I've briefly been touching/pondering the idea if we somehow can use Colin Walters OStree to our advantage ( https://live.gnome.org/OSTree )
|
16:41
|
| adamw
|
i remember reading his blog post on it and thinking 'hmm, that's interesting', but i didn't really have any concrete ideas
|
16:42
|
| nirik
|
so, httpd logs are kept for iso downloads. What info from there would you find useful?
|
16:43
|
| adamw
|
i think viking was curious about how much the nightlies are downloaded?
|
16:43
|
| nirik
|
4569 downloads in 2012-12
|
16:44
|
| Viking-Ice
|
adamw, yeah that's where I'm at came across it looks interesting wondering if we can take some kind of advantage of it but nothing concrete yet
|
16:44
|
| Viking-Ice
|
nirik, each release?
|
16:44
|
| nirik
|
5117 in 2013-01
|
16:44
|
| Viking-Ice
|
or total
|
16:44
|
| nirik
|
total
|
16:44
|
| nirik
|
any ".iso " download
|
16:44
|
| nirik
|
nightlys are only kept for a week or so tho.
|
16:45
|
| Viking-Ice
|
I'm just trying to asses the benefit of using it
|
16:45
|
| Viking-Ice
|
vs overhead
|
16:45
|
| adamw
|
the overhead's pretty tiny
|
16:45
|
| adamw
|
i think it's just nirik firing a script
|
16:45
|
| nirik
|
yep.
|
16:46
|
| adamw
|
if it doesn't build, we don't try and fix it
|
16:46
|
| nirik
|
and we keep wanting to automate it.
|
16:46
|
| Viking-Ice
|
makes sense
|
16:46
|
| nirik
|
it's useful also for spins folks to test if they ever do
|
16:46
|
| satellit
|
+1
|
16:46
|
| Viking-Ice
|
technically gnome users should be testing the gnome spin as well while we try to focus our energy on the core function
|
16:47
|
| Viking-Ice
|
but yeah
|
16:47
|
| Viking-Ice
|
how much testing did other then the *DE spin get
|
16:48
|
| Viking-Ice
|
I think those might be getting little to no testing even from their maintainers
|
16:49
|
| nirik
|
I don't think there's any reasonable way to quantify that. ;)
|
16:49
|
| adamw
|
in f18 not a huge lot, for f15->f17 i tried to get decent amount of testing for the non-blocking spins
|
16:49
|
| * nirik did in fact test the Xfce spin a number of times.
|
16:49
|
| adamw
|
we at least made sure the whole desktop matrix was done once or twice at each milestone
|
16:49
|
| adamw
|
for xfce and lxde
|
16:49
|
| adamw
|
satellit tests sugar quite a lot
|
16:50
|
| Viking-Ice
|
I'm not worried about the *DE spins ( and sugar ;) ) they all have active communities it's the other ones that concern me
|
16:50
|
| satellit
|
I also do VirtualBox installs from spins to test yum installs of other DE's with sugar
|
16:50
|
| adamw
|
outside of the desktops and sugar, hell if i know.
|
16:51
|
| adamw
|
you may well be right that they don't get much of a look.
|
16:51
|
| Viking-Ice
|
I'm wondering if we should not come up with a test matrix for those that the spin maintainers have to walk through and "pass" before release
|
16:51
|
| * nirik nods. Suggested as much to the spins list a while back.
|
16:51
|
| adamw
|
i don't mind the idea in theory, as it does kinda suck when we ship stuff that's utterly borked
|
16:51
|
| adamw
|
even if it's a spin we explicitly don't support
|
16:52
|
| adamw
|
maybe you two could get together and re-propose it to spins?
|
16:52
|
| Viking-Ice
|
what's releng take on something like that
|
16:52
|
| Viking-Ice
|
( anything we might handout at various events needs to be thoroughly tested )
|
16:53
|
| nirik
|
the spins setup is disfunctional, but attempts to fix it haven't met with anything concrete.
|
16:53
|
| nirik
|
cwickert would be the one to involve in those discussions
|
16:53
|
| cwickert
|
?
|
16:54
|
| nirik
|
cwickert: spins process... didn't go so well last cycle. ;(
|
16:54
|
| cwickert
|
yes, I know
|
16:54
|
| Viking-Ice
|
cwickert, to bring you up to speed qa/releng requesting test matrix spin has to pass before being released
|
16:55
|
| cwickert
|
but I'm afraid it will become worse when we kill the spins
|
16:55
|
| adamw
|
i don't think we hand out anything but the multi-install and multi-desktop
|
16:55
|
| adamw
|
and the regular install / desktop of course
|
16:55
|
| cwickert
|
Viking-Ice: that means what exactly?
|
16:55
|
| nirik
|
I was thinking a 2 person checkoff of a test matrix for each spin we want to promote on spins.fedoraproject.org. The rest can exist, just in a corner of alt.
|
16:56
|
| Viking-Ice
|
cwickert, test matrix spin maintainers have to walk through which ensures atleast no surprises
|
16:56
|
| Viking-Ice
|
for their spins
|
16:56
|
| adamw
|
this is the proposal
|
16:56
|
| cwickert
|
Viking-Ice: argh
|
16:57
|
| Viking-Ice
|
cwickert, the *de spins are not much worries since those have active community's it's the other spins
|
16:57
|
| cwickert
|
Viking-Ice: please consider me as an idiot who doesn't have a clue what a "test matrix spin maintainer" is
|
16:57
|
| adamw
|
cwickert: there should've been some punctuation or grammar in there :)
|
16:57
|
| Viking-Ice
|
uhum yes
|
16:58
|
| cwickert
|
I know what a test is, I know what a test matrix is, I know what a maintainer is
|
16:58
|
| adamw
|
the idea is that there would be *a* test matrix (basically just a test plan) that spin maintainers have to run through to have their spin 'approved' or whatever for a release, just a very basic 'does it boot?' smoke test
|
16:58
|
| nirik
|
I think we are talking about "it boots, selinux is enforcing and works, it lets you login, etc"
|
16:58
|
| cwickert
|
but who is supposed to maintain the test matrix for a spin?
|
16:58
|
| adamw
|
cwickert: it'd be a generic one i think
|
16:58
|
| cwickert
|
can there be specific tests for a spin?
|
16:59
|
| cwickert
|
who is to maintain them and so on
|
16:59
|
| cwickert
|
there is tons of questions
|
16:59
|
| adamw
|
i think this idea would just be a very basic generic 'smoke test'
|
16:59
|
| adamw
|
spin-specific tests are possible but would be a different thing
|
16:59
|
| Viking-Ice
|
qa would maintain the matrix I suppose we already have criteria for "core" the rest is just packages on top of that
|
16:59
|
| adamw
|
we actually already have one such matrix for the security lab spin (though no-one ever runs it)
|
16:59
|
| cwickert
|
ok, I'm sorry, I need to stop here, FAMSCo meeting
|
16:59
|
| cwickert
|
but we DO need to talk about this
|
16:59
|
| adamw
|
anyway, it seems like a decent idea
|
16:59
|
| * nirik nods.
|
17:00
|
| adamw
|
#action viking-ice to discuss the 'smoke test for spins' idea further with nirik and cwickert
|
17:00
|
| cwickert
|
I know the spins went badly
|
17:00
|
| nirik
|
yeah, I don't think anyone disagrees... just how we improve them. ;)
|
17:00
|
| cwickert
|
but on the other hand I am very frustrated about getting little or no feedback from QA about my requests
|
17:00
|
| adamw
|
sorry, which requests?
|
17:00
|
| Viking-Ice
|
yeah I missed those to
|
17:01
|
| cwickert
|
adamw: changelogs in the announcements for the differenc milestones, better browsability in the wiki, meaningful renaming of the tracker bugs
|
17:01
|
| adamw
|
oh, those
|
17:01
|
| cwickert
|
are you coming to devconf?
|
17:02
|
| adamw
|
1) i talked to andre about that one and we edited the text of the announcements somewhat to make it clearer that the 'changelog' is in the trac ticket
|
17:02
|
| adamw
|
2) unfortunately didn't get to that one yet
|
17:02
|
| adamw
|
3) we did that
|
17:02
|
| cwickert
|
I'd appreciate if we can discuss some things
|
17:02
|
| * Viking-Ice still lost...
|
17:02
|
| adamw
|
cwickert: nope
|
17:02
|
| Viking-Ice
|
I will be there kparal as well
|
17:03
|
| adamw
|
Viking-Ice: these are requests from some time back, i don't recall exactly what the forum was but i recall the discussion now
|
17:03
|
| adamw
|
cwickert: right, you can talk to viking and kparal there (probably also jskladan)
|
17:03
|
| Viking-Ice
|
perhaps this should end up in our trac instance
|
17:03
|
| adamw
|
cwickert: did you miss the tracker bug renaming thing? cos that was a whole thing a few weeks back.
|
17:03
|
| kparal
|
in a pub :)
|
17:03
|
| adamw
|
i think it may well be there
|
17:03
|
| Viking-Ice
|
kparal, with rotten shark bits ;)
|
17:03
|
| adamw
|
i'm pretty sure i filed tickets at the time
|
17:03
|
| kparal
|
Viking-Ice: that's not really a czech speciality
|
17:04
|
| adamw
|
https://fedorahosted.org/fedora-qa/ticket/273 is the ticket for the 'browsability' thing
|
17:04
|
| adamw
|
https://fedorahosted.org/fedora-qa/ticket/272 is for the 'changelog' thing...it's not that we didn't give you any feedback, really, but andre didn't entirely agree with the proposal...
|
17:05
|
| Viking-Ice
|
307 for some of it?
|
17:05
|
| adamw
|
that wasn't part of cwickert's request, no. but i do need to finish that up. sigh
|
17:05
|
| adamw
|
so much stuff to do
|
17:05
|
| adamw
|
anyhoo, we're a bit over time
|
17:06
|
| Viking-Ice
|
kparal, I will be bringing a box of bits for people to try as requested ;)
|
17:06
|
| Viking-Ice
|
yup
|
17:06
|
| adamw
|
so let's wrap up
|
17:06
|
| cwickert
|
adamw: 3) we did that?
|
17:06
|
| adamw
|
cwickert: the tracker bug renaming,.
|
17:06
|
| cwickert
|
adamw: I don't think so
|
17:06
|
| cwickert
|
what are the names now?
|
17:06
|
| adamw
|
cwickert: https://lists.fedoraproject.org/pipermail/test/2013-January/113405.html
|
17:07
|
| kparal
|
Viking-Ice: let's hope it's not an attempt to wipe out Brno's Red Hat office :-)
|
17:07
|
| cwickert
|
I am searching the wiki for 5 minutes now for the NTH have bugs :(
|
17:07
|
| adamw
|
cwickert: they're called FreezeException now
|
17:07
|
| adamw
|
cwickert: and they're always listed at https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Trackers
|
17:07
|
| adamw
|
which is linked from https://fedoraproject.org/wiki/Blocker_Bug_FAQ and https://fedoraproject.org/wiki/QA:SOP_blocker_bug_process
|
17:08
|
| * adamw sets fuse
|
17:09
|
| * cwickert needs to bail out for the FAmSCo meeting
|
17:10
|
| adamw
|
cwickert: let us know what you think about the new names
|
17:11
|
| cwickert
|
adamw: I made a proposal, so I probably prefer what I proposed, right?
|
17:12
|
| adamw
|
well i'd *hope* not everyone thinks that way :)
|
17:12
|
| adamw
|
i had a proposal too, and so did tflink, but we both prefer the final scheme
|
17:12
|
| cwickert
|
I mean, the new names are better than the old ones, but still I consider mine better :P
|
17:12
|
| adamw
|
anyhoo
|
17:12
|
| adamw
|
time to end this nightmare!
|
17:12
|
| adamw
|
thanks for coming folks
|
17:12
|
| adamw
|
#endmeeting
|
17:12
|
