QA/Meetings/20130701

From FedoraProject

Jump to: navigation, search

Contents

Attendees

  • adamw (143)
  • dan408\ (77)
  • nirik (35)
  • jreznik (32)
  • tflink (28)
  • cwickert (14)
  • Southern_Gentlem (5)
  • brunowolff (5)
  • zodbot (4)
  • Martix_ (4)
  • dgilmore (4)
  • robatino (3)
  • Cerlyn (2)
  • mkrizek (1)
  • pschindl (1)

Agenda

  • Fedora 19 retrospective and wrap-up
  • Fedora 20 planning
  • Taskbot
  • Test Days
  • Open floor

Fedora 19 retrospective and wrap-up

  • The Fedora_19_QA_Retrospective page is now up and ready for feedback
  • No-one had any other major post-F19 topics that weren't already in hand

Fedora 20 planning

  • nirik and dgilmore will be proposing a new policy for deciding which spins to ship for F20
  • adamw will continue with Final criteria rewrite and look at updating the validation test case set
  • With the new Changes policy, the F20 'feature list' is Releases/20/ChangeSet
  • jreznik would like to see functional validation of Changes for F20

Taskbot

  • The current AutoQA setup is running on F17 which will go EOL soon: we may need to spend time moving it to something newer
  • tflink says we're hoping to have a functional taskbot in place during F20 cycle

Test Days

  • Special out-of-cycle request
  • Deferred till next week

Open floor

  • The spin-kickstarts package build process has been improved, which may make it less of a pain to do package updates

Action items

  • adamw to add an item for Change validation discussion to next week's agenda

IRC Log

adamw #startmeeting Fedora QA meeting 15:01
zodbot Meeting started Mon Jul 1 15:01:54 2013 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot. 15:01
zodbot Useful Commands: #action #agreed #halp #info #idea #link #topic. 15:01
adamw #meetingname fedora-qa 15:02
zodbot The meeting name has been set to 'fedora-qa' 15:02
adamw #topic Roll call 15:02
* pschindl is here 15:02
* tflink is here 15:02
dan408\ hi 15:02
* mkrizek is here 15:02
* Cerlyn is here 15:03
adamw set phasers to FUN 15:03
* jreznik is there, would like to leave a bit earlier, so go on, go on! 15:03
* brunowolff is here 15:03
adamw #topic Fedora 19 retrospective and wrap-up 15:04
adamw so, we got F19 tested and out the door with only a one week slip - big congratulations to everyone! 15:04
adamw in case anyone missed the ML message: 15:05
jreznik yep, thanks everyone! 15:05
adamw #info I put the retrospective page up recently at https://fedoraproject.org/wiki/Fedora_19_QA_Retrospective 15:05
adamw you can add feedback on the *QA* process to that page 15:05
Southern_Gentlem adamw, my question is how the sudo switch is missing from the gnome install 15:05
adamw Southern_Gentlem: the GNOME team does not want it to be optional. 15:06
adamw it's by design for that tool. 15:06
adamw that doesn't quite tie in with anaconda views it, but then i don't think we're quite at the point where GNOME gets to define the UI of *the entire distribution* :) 15:06
Southern_Gentlem so first boot you have to give admin priviledges to the first user 15:06
adamw if GNOME was designing the installer, it wouldn't have a switch either: the first user account would always be an admin. 15:06
Southern_Gentlem something wrong with that 15:07
adamw if you install GNOME and don't create a user in anaconda, yes. 15:07
* dan408\ sips coffee 15:07
adamw so let's see, as well as general f19 stuff, some leading questions... 15:07
adamw anyone particularly worried about any specific bugs that slipped through the net and have ideas to do anything about them? 15:08
tflink other than the mac bug? 15:08
dan408\ no 15:09
tflink nvm, already on the list 15:09
adamw including that, if we have anything productive to contribute 15:09
adamw though personally i'm not hugely concerned about that one 15:09
dan408\ it's good ol' fedora 15:09
tflink yeah, I don't have anything to contribute that hasn't already been said 15:10
adamw can anyone think of any issues we need to document that aren't documented already? 15:10
tflink I'm minorly concerned about what criteria the cloud folks want to add, but that can wait until they're proposed 15:11
robatino i'm already seeing bugs 946964/955779 on one of my newly installed F19 boxes. i think it may be a common issue 15:11
dan408\ nothing really, same usual questions popping up in #fedora 15:11
dan408\ grub questions, video drivers, getting up and running 15:12
dan408\ these are pretty much documented 15:12
adamw cool 15:12
adamw so no big new stuff 15:12
dan408\ no. 15:13
dan408\ i really dont see anything major at the moment 15:13
adamw robatino: if you can clarify that one it could certainly be a commonbugs candidate, but I haven't had issues using gdm in KVM virt machines 15:13
adamw thanks dan 15:13
dan408\ np 15:14
robatino i'm seeing it on bare metal now (on a machine with blacklisted 82865 video) 15:14
robatino i'll be investigating more 15:14
adamw for the record, i've done most of the obvious commonbugs entries, the ones remaining are mostly ones that aren't entirely clear, but i'll take another pass through the list today 15:15
adamw anything more for f19 or shall we move on to the next one? :) 15:15
jreznik thanks adamw for commonbugs 15:15
dan408\ the next one ? 15:15
* dan408\ gets scared 15:15
adamw f20! 15:16
dan408\ NO 15:16
dan408\  :( 15:16
dan408\ 2 weeks vacation for all 15:16
adamw #info Fedora 20 is cancelled, dan408's orders 15:16
dan408\ not cancelled 15:16
adamw the official QA dictionary does not know the meaning of the word 'vacation' 15:17
brunowolff I added a note about getting a TC for the minor spins. 15:17
dan408\ alright 15:18
adamw brunowolff: that's really up to releng 15:18
adamw we don't order them what to build 15:18
adamw we just ask for a tC 15:18
jreznik adamw: vacation? is it somehow related to vaca in spanish? :D 15:18
adamw #topic Fedora 20 planning 15:19
brunowolff It affected QA of those spins. 15:19
adamw so F19 is SO last week 15:19
adamw brunowolff: well, true. 15:19
* nirik would like to clearly propose dgilmore's spins critera for the f20 cycle. 15:19
adamw nirik: can you expand on what you mean by that? 15:19
nirik for tc/rc's we compose all approved spins, but spins must get 2 testers to ack that they passed basic tests. Those that do, are released for that milestone. Those that don't aren't. 15:20
jreznik for f20 schedule reference - https://fedoraproject.org/wiki/Releases/20/Schedule (see, no earlier than) 15:20
nirik if something doesn't get any acks for a cycle, it's dropped. 15:20
dan408\ can these acks be done via email? 15:21
* nirik was going to write up something for devel/test/spins 15:21
adamw dan408\: good question 15:21
nirik whatever folks would prefer, sure. 15:21
jreznik nirik: well, we planned it in milano with cwickert too (similar, even for the big spins) - so I'm +1 for that 15:21
nirik but a wiki might be easier to track 15:21
dan408\ +1 to a wiki on that 15:21
adamw we have matrices for desktop spins, we don't print a matrix for every other spin 15:21
jreznik nirik: or real ticket... 15:21
nirik yeah, could get confusing tho... with all the spins... 15:22
jreznik or matrix... 15:22
adamw a page that just had 'does it boot?' for every spin would be easy to write, i guess. 15:22
* nirik doesn't care how we track it really. 15:22
adamw does anyone see a major problem with the policy in general? 15:22
adamw from a QA perspective? doesn't seem like a problem for us, to me 15:22
nirik I was thinking: boots, can login, network works? (or any other basic things) 15:22
dan408\ as long as we aren't dropping spins left and right due to lack of acks, im +1 15:22
nirik well, I suspect we will, but we will see. 15:23
adamw i suspect the *two* tester requirement may be too many 15:23
adamw we don't get two testers for KDE sometimes, enver mind anything else 15:23
nirik should it be 1/ 15:23
nirik  ? 15:23
adamw 1 seems more realistic 15:23
cwickert hold on 15:23
cwickert It depends on what you want to test 15:23
adamw and for a really basic functionality check, i don't see the need for double testing 15:24
nirik anyhow, I am not saying we approve anything. I am just saying I intend to start a clear dialog on this for f20. ;) 15:24
adamw cwickert: "<nirik> I was thinking: boots, can login, network works? (or any other basic things)" 15:24
adamw nirik: sure, understood 15:24
dgilmore adamw: my thought was the spin maintainer and one other 15:24
cwickert most spins are based on other spins, so they should have pretty good coverage already 15:24
adamw #info nirik and dgilmore will be proposing a new policy for deciding which spins to ship for F20 15:24
cwickert like the jam-kde spin inherits testing from KDE 15:25
adamw cwickert: the problem is that just adding packages can bust stuff sometimes, we can't assume a pass for the KDE spin means a pass for all KDE-derived spins 15:25
jreznik dgilmore: spin maintainer + one other makes sense for me 15:25
nirik except when it doesn't compose or has other issues. 15:25
adamw jreznik: i think sometimes, 'spin maintainer' may be all we can get. 15:25
cwickert all we then need to test is: 1) it composes, 2) it boots, 3) one can login and 4) all applications run as expected 15:25
dgilmore adamw: i know in the past they have not tested 15:25
adamw 'all applications run as expected' is a large bear trap 15:25
cwickert and this should IHNO be done by two people 15:25
cwickert owner and one more 15:25
adamw it takes about three hours to run that test for KDE 15:25
dan408\ +1 cwickert 15:26
adamw and it's rarely a 100% pass 15:26
nirik even spin owner is better than what we have now. 15:26
cwickert s/IHNO/IHMO 15:26
dan408\ it takes 3 hours to run that test for kde? 15:26
adamw yup 15:26
dan408\ is kde really that slow? 15:26
nirik yeah, all applications is a bit crazy... 15:26
adamw just launching every damn app in the menus and testing it can do _something_ 15:26
dan408\ or are you talking about KDE + derivative spins? 15:26
cwickert adamw: I don't expect it to pass 100, I just expect it to be tested 15:26
adamw dan408\: no, it just has a lot of apps 15:26
jreznik dan408\: manually, yes - there are a lot of apps, you have to try to click over, load document 15:26
adamw dan408\: Desktop has rather fewer, still takes 1-2hrs though 15:26
Cerlyn testing all apps on something like the security spin would be difficult 15:26
cwickert adamw: why would it take 3 hours to test kde? 15:26
dan408\ jreznik: that's what we have you for 15:27
jreznik definitely time to talk to vhumba about this one 15:27
nirik I'd be fine with that as an optional. 15:27
cwickert adamw: you mean according to the criteria I just outlined? 15:27
dan408\ well KDE is a "release blocking" desktop 15:27
adamw cwickert: "all applications run as expected" 15:27
adamw anyhow 15:27
adamw we don't need to get bogged down in the details 15:27
dan408\ but for a basic spin you dont need to "launch every app" 15:27
adamw yeah, that's kinda what I was saying, i don't think that test is even required 15:27
adamw maybe check the key apps for the spin, sure 15:27
dan408\ yes. 15:28
dgilmore so long as people can get in, get online and open a browser is a good start 15:28
dan408\ i.e. can you open a "konsole" or a "terminal" 15:28
dgilmore that the apps work as expected is a plus 15:28
* nirik is with dgilmore. We should walk before we run. ;) 15:28
dan408\ okay 15:28
dan408\ i think we are all talking about the same thing here 15:28
adamw yeah, i think dan's right 15:29
adamw no-one seems to think the idea is a screaming failure, so let's just wait for the formal proposal and we can bikeshed the details then 15:29
adamw everyone get your preferred tin of paint ready 15:29
cwickert adamw: so, how many applications are there in the menu? say 50, starting one takes 10 seconds. that makes it less than 10 minutes. you just fire them up, and see if they start, you don't test each and every menu option 15:29
brunowolff There are other details regarding arch and whether the image is optical, dd'd to usb or lcti'd to usb that should be considered. 15:30
adamw cwickert: if you literally just want to see if they run, it might take a bit less. if you want to test at least that you can do some basic operation without it crashing, much longer. 15:30
* nirik notes f19 xfce fails on several apps in that test right now. ;) 15:30
* cwickert knows 15:30
dan408\ heh 15:30
cwickert LXDE has two fails 15:30
* adamw waves the 'next topic' hammer threateningly 15:31
dan408\ maybe there should be a recompose of the live spins 15:31
dan408\ but i think someone is waving a hammer so i should run 15:31
Southern_Gentlem a reduction in live spins 15:31
Southern_Gentlem desktops and provide ks for rest 15:32
adamw Southern_Gentlem: i don't think there's anything to be gained by arguing about that in a QA meeting. 15:33
adamw alright 15:33
adamw so other than this proposal, what do we need to do for F20? 15:33
dan408\ uh 15:33
nirik nothing, lets ship it! :) 15:33
adamw i have a few things lined up: I need to continue revising the final criteria (which got stalled during final validation), and i'd like to go through the whole validation test case set 15:33
dan408\ how are we pracitcally dealing with "features" 15:33
* nirik runs 15:33
dan408\ nirik: you have to compose it first 15:33
nirik there are no more features. There are now changes. ;) 15:34
adamw #info adamw will continue with Final criteria rewrite and look at updating the validation test case set 15:34
dan408\ nirik: right 15:34
adamw dan408\: is the question 'how do things change for QA with the new Changes process'? 15:34
adamw if so, excellent question 15:34
dan408\ yes 15:34
dan408\ sure 15:34
adamw nirik: can you help us with that? 15:35
* dan408\ is trying to help with a "change" for 20 15:35
nirik I could try. ;) I don't think things change too much aside from that the 'standalone' ones vs system wide might help with what you need to test? 15:35
dan408\ i guess for me it was more of a better understanding, and now i guess it's more for adamw to write criteron for standalone vs systemwide changes? 15:36
adamw i don't know if there's some kind of obvious criterion 15:37
adamw and i'm not the only one who can write criteria :) 15:37
dan408\ there is 15:37
adamw if you think we need one, draft one up 15:37
dan408\ so i guess there should be a decision of whether certain systemwide changes should be "release blocking" or not 15:37
adamw the distinction between 'standalone' and 'systemwide' should give us some clues of what to test 15:37
dan408\ fesco is no longer voting on these things anymore 15:38
adamw well, what would it mean for a Change to be 'release blocking'? 15:38
dan408\ they are just proposed 15:38
dan408\ adamw: Well, do we plan any more anconda "improvements" for 20? 15:38
dan408\ Gnome 3.10 would be a systemwide change wouldn't it? 15:38
* jreznik would be glad so see qa involved in change process - especially what we miss now is the final ON_QA validation step - especially for the bigger system wide ones... 15:39
dan408\ Well what fesco used to vote on, who votes on it now? QA? 15:39
adamw as in voting on whether features are 'approved'? 15:39
jreznik dan408\: FESCo still votes on both 15:39
* nirik gets back. what? 15:40
dan408\ thank god 15:40
adamw nirik: who's right, jreznik or dan? 15:40
jreznik just for systemwide - FESCo wants change by change, for selfcontained - in batch 15:40
nirik right. 15:40
nirik actually I thought we were just approving selfcontained unless someone asked for a vote. 15:40
nirik but either way. 15:40
jreznik if somebody is not ok with selfcontained being selfcontained - it could be raised to fesco to vote on as nirik says 15:41
* dan408\ thinks all systemwide changes should be approved by fesco 15:41
nirik all of them go to devel list for comment. 15:41
adamw i think that's clearly the case 15:41
nirik all systemwide ones must be passed by fesco 15:41
adamw okay 15:41
dan408\ then self contained can just go to devel list 15:41
adamw so we don't need to try and take that function over or anything, dan. but thanks for raising it 15:41
dan408\ right because there's obviously confusion there 15:42
adamw dan408\: i think everyone's clear that systemwide Changes are reviewed, though. which is the important thing 15:42
dan408\ so i guess once that's done 15:42
adamw we should certainly read the threads on devel@ and point out concerns we have with Changes from a QA perspective, btw 15:42
dan408\ we can get to testing these things 15:42
adamw i've been doing that, but it'd be great if others can too 15:42
dan408\ well jreznik needs to send out the list.. 15:43
dan408\ i kind of made a copy for 20 out of panic 15:43
jreznik adamw: yes, that's one important thing from qa perspective 15:43
adamw dan408\: each systemwide Change should show up as its own thread on devel@ 15:44
jreznik adamw: and that other part is validation of at least of systemwide changes 15:44
dan408\ http://fedoraproject.org/wiki/Releases/20/FeatureList 15:44
jreznik aka is it really implemented? 15:44
dan408\ there is nothing up there 15:44
adamw well, that gets back to the question I asked Dan above 15:44
jreznik dan408\: it's not used anymore 15:44
adamw what does 'validation' of a feature mean exactly? 15:44
nirik we should remove/fix that page 15:44
dan408\ jreznik: which is bad imo 15:44
adamw that the feature works 100%? that it works at all? that it doesn't cause breakage in anything else? what? 15:44
dan408\ how do we track this now 15:44
nirik there will be another page without the word "Feature" in it? 15:45
adamw good question dan 15:45
jreznik dan408\: it's going to be replaced by https://fedoraproject.org/wiki/Releases/20/ChangeSet - being, sorry, only two accepted changes and I have to change my script to generate right list 15:45
jreznik nirik: yes 15:45
adamw nirik: i'd expect the 'FeatureList' location to re-direct to the new list for at least a few releases 15:45
dan408\ why are we confusing the hell out of people? 15:45
adamw i don't know about anyone else, but XX/FeatureList is in *my* muscle memory 15:45
jreznik adamw: of course I can do it 15:45
nirik and possibly not in wiki... up to jreznik 15:45
dan408\ adamw: +1 15:46
dan408\ that is why i basically copied the wiki page at the time 15:46
dan408\ http://fedoraproject.org/wiki/Releases/19/FeatureList 15:46
dan408\ this is beautiful 15:46
jreznik adamw: the idea was to extend the list with more info - not only a list but a place with details - see the ChangeSet template... there should be docs, marketing and I'm more than happy to track QA status there too 15:46
adamw redirect to Releases/20/ChangeSet would be fine. 15:46
jreznik adamw: ok, I redirect from old policy and yeah, I'll do it for a few releases too 15:47
adamw so my take on 'validating features' has always been that, well, we validate the release 15:47
adamw features are a part of the release 15:47
jreznik for FeatuesList/ChangeSet 15:47
adamw if they break something important, our validation tests should catch that 15:47
jreznik adamw: release is validated, does it mean some changes are really implemented? no 15:47
dan408\ I guess we will repropose E for f20 as a "self contained" feature 15:47
adamw but i'm not sure if that philosophy is water-tight; we might want to reconsider 15:48
jreznik dan408\: do it please and let me knwo 15:48
dan408\ let's make sure that whatever is listed under the features category is moved to the changes category 15:48
adamw jreznik: so, you want us to test that the Changes all actually work? that's another bunch of work for us to do, i guess 15:48
dan408\ #link http://fedoraproject.org/wiki/Features/Enlightenment 15:48
jreznik adamw: one thing is - we know release is ok, but we are never 100% sure if all changes are ok and you know, we do release announcements (and many times we realized hey, it's actually not there etc.) 15:49
adamw #info jreznik would like to see functional validation of Changes for F20 15:49
adamw volunteers welcome... 15:49
jreznik adamw: that's the question if it's doable... 15:49
adamw well, we can look at that 15:49
adamw for now let's move on as time is getting short 15:49
tflink yeah, that sounds like a lot of additional work 15:49
dan408\ jreznik: I guess you just need to check http://fedoraproject.org/wiki/Category:FeatureReadyForWrangler 15:50
jreznik tflink: it is but it's something we miss now :( 15:50
adamw #action adamw to add an item for Change validation discussion to next week's agenda 15:50
adamw let's discuss it in more depth next week 15:50
jreznik dan408\: I'm in touch with owners and we're migrating it from features to changes 15:50
adamw #topic Taskbot 15:50
dan408\ jreznik: okay 15:50
adamw wanted to check in on the current status of taskbot 15:51
adamw #chair tflink 15:51
zodbot Current chairs: adamw tflink 15:51
adamw what's the news, flinkonian one? 15:51
tflink not much to say, getting ramped up now that F19 is mostly over 15:51
tflink I want to get something in place before F20 starts up 15:52
adamw that'd be nice 15:52
tflink but most of AutoQA is running on F17, so that may need fixing sooner than later 15:52
adamw ah, yes 15:52
adamw #info current AutoQA setup is running on F17 which will go EOL soon: we may need to spend time moving it to something newer 15:52
tflink the fedora version doesn't really matter much for our current tests 15:53
tflink but it seems like bad form to be running on an EOL version 15:53
dan408\ well time to upgrade to 19 or "rawhide" ? 15:53
tflink yeah, it's just a lot of work 15:54
adamw i'm not sure if it's worth wasting a lot of work for 'bad form' 15:54
adamw if there's no actual security issue to running an EOL'ed Fedora for the test machines... 15:54
tflink they're all behind a firewall 15:54
dan408\ do they have selinux enabled? :P 15:55
tflink everything publicly accessible is running el6 15:55
tflink dan408\: not sure I see how that matters 15:55
tflink they don't even have public ips 15:55
adamw just a joke, i think 15:55
dan408\ kind of a joke 15:55
adamw so, i'd consider that question before spending a bunch of time on it 15:55
adamw especially if there is a realistic road ahead to taskbot in a reasonably short timeframe and this would delay that 15:55
tflink yeah, I was hoping to avoid upgrading if we can 15:55
adamw maybe work with infra to see what they think 15:56
tflink about the upgrade? 15:56
adamw yeah 15:56
tflink that's all us 15:56
adamw if they can think of an actual practical reason we may need to do it 15:56
adamw ah okay., 15:56
tflink but it would be worth asking if they can see an issue with the plan 15:56
adamw right, more input always valuable 15:57
adamw as for taskbot, what were you hoping to get in place? 15:57
tflink I 15:57
tflink I'd like to get something mostly equivalent to AutoQA soon 15:58
nirik reinstalling thing should be pretty easy I would think... but happy to discuss details. 15:58
tflink nirik: there are some complications in doing that which lead back to the mess that is the method we use for sysadmin 15:58
adamw well, that'd sure be nice if we can do it 15:58
adamw is there stuff that people can help out with? I see john dulaney has been trying to help with depcheck testing 15:59
nirik tflink: yeah, lets talk in admin later and hash out a plan. 15:59
* tflink has another meeting @ the top of the hour 15:59
tflink at the moment, not much - there are still way too many variables 15:59
adamw okay 15:59
* nirik nods. Later is just fine. 15:59
adamw we can move the test day topic to next week 16:00
tflink ideas for tests, or even better - code 16:00
Martix_ adamw: I'm here 16:00
Martix_ adamw: but I'll be here also next week ;-) 16:00
adamw #info we're hoping to have a functional taskbot in place during F20 cycle 16:00
adamw Martix_: the date on the special request wasn't super soon, was it? 16:00
Martix_ adamw: what special request? 16:01
adamw it isn't, okay. 16:01
adamw let's do that next week 16:01
Martix_ ok :-) 16:01
adamw #topic open floor 16:01
tflink wait, I got the time wrong on the next meeting. oh well not much to say right now, anyways 16:01
adamw anyone have anything important we didn't cover? 16:01
dan408\ i think we're good 16:02
jreznik no blocker meeting follow up today? :) 16:02
adamw f20 blockers, sure :) 16:02
* adamw found one yesterday 16:02
dan408\ ack 16:02
brunowolff Semi related to QA is that spin-kickstarts has been changed to be easier to build. 16:03
adamw awesome, thanks bruno 16:04
dan408\ yup saw your email on that 16:04
adamw #info spin-kickstarts package build process has been improved, which may make it less of a pain to do s-k updates 16:04
adamw anything else, folks? 16:06
* adamw attaches Quantum Fuse to Cat 16:06
tflink that brings up bad images 16:06
adamw =) 16:07
adamw okay, thanks for coming out everyone 16:07
adamw #endmeeting 16:07

Generated by irclog2html.py 2.12.1 by Marius Gedminas - find it at mg.pov.lt!