From Fedora Project Wiki

< QA‎ | Meetings

Revision as of 18:48, 7 February 2011 by Jlaska (talk | contribs) (Updated notes)

Attendees

People present (lines said)

  1. jlaska (174)
  2. adamw (112)
  3. kparal (60)
  4. Viking-Ice (45)
  5. rbergeron (30)
  6. maxamillion (13)
  7. brunowolff (11)
  8. vhumpa (10)
  9. tflink (8)
  10. wwoods (2)
  11. robatino (2)
  12. Southern_Gentlem (1)
  13. CRCinAU (1)

Unable to attend:

  1. Rhe (hopefully sleeping)
  2. Hongqing (hopefully sleeping)

Agenda

Previous meeting follow-up

  1. jlaska - update https://fedoraproject.org/wiki/Proven_tester#Mentoring_process with information about how to handle FAS group requests without a corresponding ticket

AutoQA update - autoqa-0.4.4 and DevConf + FUDCon updates

Owner
kparal
Summary
jlaska created a great patch for autotest to automatically transfer all autoqa config files from the server to the client
wwoods improved his depcheck test and sent us final version proposal
jskladan posted final version of his new_koji_watcher code
kparal prepared a patch for upgradepath to work with the new post-bodhi-update-batch event (part of the new_koji_watcher stuff)
jskladan created new milestone in AutoQA trac - Finger Food - containing very small tasks suitable for AutoQA newbies -- https://fedorahosted.org/autoqa/milestone/Finger%20Food
kparal created new documentation AutoQA Development -- https://fedoraproject.org/wiki/AutoQA_Development
FUDCon:Tempe_2011 AutoQA slides available at https://fedoraproject.org/wiki/QA:Presentations
Next Steps ...
Complete packaging of compat-Django-1.0.4 security patches and submit for review (jlaska)
Wwoods to review current depcheck patch, and provide feedback
Continue testing and merging of new koji-watcher

Tue, Feb 08 - Alpha 'Test Compose'

Owner
rhe
Summary
See task#19 on Fedora 15 QA calendar (http://poelstra.fedorapeople.org/schedules/f-15/f-15-quality-tasks.html)
Next Steps ...
  1. jlaska - ask rel-eng to file tickets for upcoming Alpha milestones
  2. jlaska - ping clumens about a new anaconda build for the alpha TC
  3. rhe or robatino - Announce ISO availability, and commence testing

Thu, Feb 10 Test Day -- FreeIPA v2

Owner
adamwill
Summary
See https://fedorahosted.org/fedora-qa/ticket/163
No updates in ticket or on wiki yet
Next Steps ...
  1. adamw - reach out to dmitri to check-in on FreeIPA test day prep

Fri, Feb 11 - Alpha Blocker Meeting (f15alpha) #3

Owner
jlaska
Summary
The 3rd scheduled blocker bug meeting will be hosted this Friday (see https://fedoraproject.org/wiki/QA:SOP_Blocker_Bug_Meeting).
The f15 Alpha blocker bug is https://bugzilla.redhat.com/show_bug.cgi?id=F15alpha. If you think a bug should block the release of F15 Alpha, set it as blocking that bug, and it will be reviewed at the meeting
The F15 Alpha nice-to-have bug is https://bugzilla.redhat.com/show_bug.cgi?id=f15alpha-accepted. This is for bugs which don't block the release but for which fixes should be allowed through alpha freeze. You can propose bugs for it directly if you're sure they don't meet the blocker requirements; otherwise we usually put things on it when they don't get accepted as blockers
Next Steps ...
  1. jlaska - send F15Alpha blocker review announcement
  2. All - file bugs and propose as blockers

Testing Fedora 15 in KVM virtual machines

Owner
kparal
Summary
Kamil raised the topic on the mailing list -- see http://lists.fedoraproject.org/pipermail/test/2011-February/096856.html
After discussing the differences in environments, the team agreed that using KVM for testing is acceptable. However, testers and test organizers, should recognize that virt is only one of many environments that may need testing.
Additionally, the desktop validation matrix may need review to account for fallback mode, and other areas to ensure test coverage in multiple environments (bare-metal and virt)
Next Steps ...
  1. HELP - Adjustments to the desktop validation matrix (https://fedoraproject.org/wiki/QA:Desktop_validation_testing) may be required to accomodate for fallback mode, and/or other hardware environments. Suggestions/drafts encouraged.

Open discussion - <Your topic here>

GNOME3 Test Day Summary

Owner
Adamwill
Summary
We had the first of three GNOME 3 test days on thursday, it went very well thanks to heroic work by the desktop team to get testing packages ready
We had several dozen testers and lots of juicy bugs reported.
Thanks to everyone who came and tested!!!
rbergeron asked about the process for how bugs are prioritized after the event, how do we know the bugs are prioritized and fixed as needed. Adamwill noted that he would review the bugs filed and recommend moving to the appropriate upstream (or downstream) blocker list.
Next steps ...
Adam expecting to send test summary later today

Best Practices for test day bug reporting

Owner
johannbg
Summary
After discussing the GNOME3 test day practice of filing functionality bugs upstream (bugzilla.gnome.org), and packaging bugs downstream (bugzilla.redhat.com), some debate ensued as to whether multiple bug reporting systems was an impediment to test day contributors.
One idea was to require that all future test days file bugs in the same downstream bugzilla
One idea was to add a caution/note to the test day SOP to noting that multiple bug trackers may be confusing to participants.
The meeting ran over the alloted time, and no agreement was reached
Next steps ...
johannbg agreed to continue discussion on test@lists.fedoraproject.org

HOWTO debug guide requirement

Owner
johannbg
Summary
johannbg suggested we should make it a requirement for test days to have how_to_debug pages ready present for the components that are being tested present and ready before the test
Adamwill noted that adding a suggestion to the test day SOP for a debug guide seems like a good idea
Next steps ...
Propose update to https://fedoraproject.org/wiki/QA/SOP_Test_Day_management

Action items

  1. jlaska - ask rel-eng to file tickets for upcoming Alpha milestones
  2. jlaska - ping clumens about a new anaconda build for the alpha TC
  3. adamw - reach out to dmitri to check-in on FreeIPA test day prep
  4. jlaska - send F15Alpha blocker review announcement
  5. Viking-Ice - solicit feedback on test@lists.fedoraproject.org to see whether we need to require only bugzilla.redhat.com use during test days

IRC Transcript

jlaska #startmeeting Fedora QA Meeting 16:00
zodbot Meeting started Mon Feb 7 16:00:10 2011 UTC. The chair is jlaska. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00
zodbot Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00
jlaska #meetingname fedora-qa 16:00
zodbot The meeting name has been set to 'fedora-qa' 16:00
jlaska #topic Gathering 16:00
* kparal waves 16:00
jlaska Show of electronic hands ... 16:00
* rbergeron raises her robotic arm 16:00
vhumpa timber 16:00
jlaska vhumpa: kparal: welcome! 16:01
tflink * waves 16:01
jlaska howdy tflink  :) 16:01
jlaska rbergeron: somehow I'm picturing something from the terminator 16:01
* brunowolff is here 16:01
adamw yo 16:02
jlaska brunowolff: adamw: howdy 16:02
* Southern_Gentlem 16:02
jlaska anyone else ... robatino Southern_Gentlem Viking-Ice ? 16:02
* CRCinAU is physically here... 16:02
robatino here 16:02
vhumpa Hey everybody 16:02
rbergeron jlaska: just pretend i'm one of the girls from the terminator tv series. 16:02
rbergeron :) 16:02
jlaska be nice today folks ... this is tflink and vhumpa's first qa meeting 16:03
jlaska rbergeron: CRCinAU: greetings :) 16:03
jlaska okay, let's get this party started! 16:03
jlaska #topic Previous meeting follow-up 16:04
jlaska I don't believe a meeting was held last week ... at least I didn't see any minutes posted 16:04
jlaska and there was only one action item from the previous previous meeting 16:04
jlaska #info jlaska - update https://fedoraproject.org/wiki/Proven_tester#Mentoring_process with information about how to handle FAS group requests without a corresponding ticket 16:04
jlaska I added a blurb to the end of the mentoring section at https://fedoraproject.org/wiki/Proven_tester#Mentoring_process 16:04
adamw yay documentation 16:05
jlaska heh ... +2 sentences ... I'm certainly not a doc guru  :) 16:05
jlaska that titled goes to our very own adamw 16:05
* maxamillion is here (kinda) 16:05
* jlaska tips hat to maxamillion 16:05
adamw hey maxa 16:05
jlaska so comments welcome, otherwise I'm checking that off my list 16:06
jlaska anything else from previous meetings that we need to review? 16:06
* jlaska sets fuse at 15 seconds 16:06
adamw don't think so 16:06
jlaska alrighty ... time for the show 16:06
jlaska kparal: are you ready to kick things off today? 16:07
kparal jlaska: sure 16:07
jlaska #topic AutoQA update - autoqa-0.4.4 and DevConf + FUDCon updates 16:07
jlaska kparal: take it away my good man 16:07
kparal ok, so, this is the summary for the last two weeks 16:07
kparal #info jlaska created a great patch for autotest to automatically transfer all autoqa config files from the server to the client 16:07
kparal We already had similar hack before, but it didn't work quite right (we used to read config files from current directory). The new one solved a lot of issues for us and in the future we could use this approach for transferring also the AutoQA library (and therefore won't have to maintain the clients at all) - I already created a proposal for that, we will want to deal with it in the next+1 release. 16:08
jlaska kudos to lmr for that patch idea 16:08
kparal #info wwoods improved his depcheck test and sent us final version proposal 16:08
kparal It is not included yet, I have posted some feedback on that (there were a few tracebacks) and we haven't received a reply yet. But generally it looks in a quite good shape. 16:08
kparal #info jskladan posted final version of his new_koji_watcher code 16:09
kparal The new Koji watcher code should obsolete the old koji watcher and also the old Bodhi watcher. I would almost agree to include it in the master, but the output of the new Koji watcher and of the old Bodhi watcher still differs a little. We believe it's caused by a bug in Bodhi that should be fixed by now, but not deployed yet. Confirmation from lmacken is needed. 16:09
jlaska is wwoods lurking? not sure if that's on his radar 16:09
jlaska kparal: how much does it differ? 16:10
kparal lmacken seems to be not available right now, pitty 16:10
jlaska do we gain or lose tests using the new watcher? 16:10
kparal jlaska: well, both :) 16:10
* wwoods is lurking 16:10
jlaska kparal: heh ... isn't that always the case! :) 16:10
wwoods I'll read the followup comments and see what I can do 16:10
kparal we gained some more, which is a good thing. the old bodhi watcher was flawed 16:10
kparal but we also lost some, which is I believe caused by Bodhi bugs 16:10
kparal I need confirmation from lmacken that the fix has not been pushed to production yet 16:11
jlaska wwoods: sweet, thank you ... I've got your branch setup and can walk through some sample test runs if needed 16:11
kparal if it was, then we need to look at it more 16:11
jlaska kparal: okay 16:11
kparal and after the fix is pushed, we need to do the testing again 16:11
kparal so, lmacken needed for us to be certain :) 16:12
jlaska okay ... we'll send out a search party after meeting 16:12
kparal alright, let's move on next 16:13
kparal #info kparal prepared a patch for upgradepath to work with the new post-bodhi-update-batch event (part of the new_koji_watcher stuff) 16:13
kparal I have the code prepared, but it was not yet posted for review, because we need to accept new_koji_watcher first. So, more on this later. 16:13
jlaska points for preparedness! 16:13
kparal :) 16:13
kparal #info jskladan created new milestone in AutoQA trac - Finger Food - containing very small tasks suitable for AutoQA newbies 16:13
kparal That will surely come handy soon. 16:13
kparal #link https://fedorahosted.org/autoqa/milestone/Finger%20Food 16:13
kparal and the last one: 16:14
kparal #info kparal created new documentation AutoQA Development 16:14
jlaska I like that idea, I added a few items to that roadmap last week 16:14
kparal It describes an easy way how to setup full AutoQA development environment (your machine, AutoQA server, AutoQA client). Ideal for new guys I believe (and also for me to remember the setup). 16:14
kparal #link https://fedoraproject.org/wiki/AutoQA_Development 16:14
kparal that's all the news I believe 16:15
tflink as someone who is following that doc, would you prefer suggestions if/when I run into issues or should I just edit the wiki pages? 16:15
jlaska ooh, very handy 16:15
kparal tflink: as you seem fit, both approaches are just fine 16:15
vhumpa Looks very nice to start up, I'll check it out 16:15
tflink kparal: OK, thanks 16:15
adamw tflink: i'd say in general if you see something small that's just inaccurate / out of date, fix it 16:16
kparal tflink: you can discuss it on Talk page, at autoqa-devel, over IRC, or just edit it if you find an error :) 16:16
adamw for bigger stuff or if you're not quite sure, find an adult first :P 16:16
* jlaska looks around for one 16:16
adamw as in, someone who knows about what you're editing 16:16
adamw sometimes that may even be jlaska (terrifying as the prospect sounds( 16:17
jlaska god help us 16:17
kparal tflink: certainly don't be afraid to harass me with any autoqa questions you have, anytime 16:17
tflink adamw: you mean that making big changes without clearing it with someone would be a bad idea? :-P 16:17
adamw tflink: sometimes! 16:17
adamw :P 16:17
adamw main thing, if you're pretty sure about your change, just go ahead and do it, anything else creates unnecessary bureaucracy, the ROOT OF ALL EVIL 16:18
jlaska kparal: Using that handy openoffice.org template you used for your AutoQA presentation, I gave a brief update on the status and roadmap of autoqa at FUDCon Tempe. The slides are fairly brief, I spent more time talking and discussing with participants 16:18
tflink kparal: will do, I've been looking through the code and am just getting to the point where I can start asking intelligent questions 16:19
jlaska #info FUDCon:Tempe_2011 AutoQA slides available at https://fedoraproject.org/wiki/QA:Presentations 16:19
vhumpa kparal: i wish to get to that point by this week :-) 16:19
kparal jlaska: were there some interesting questions/responses from the audience? 16:19
jlaska I had a lot of great conversations around AutoQA and autotest as well. I'll provide more in a blog/trip_report later this week 16:20
kparal ok, nice 16:20
jlaska kparal: requests for new test wrappers for existing tests (tickets filed), dmalcolm requested his rpmlint whitelisting idea again and showed some sample code, some discussion around ensuring deps don't 'splode for some SIGs (namely the server SIG) 16:21
jlaska kparal: anything you wanted to highlight for the developer conference this weekend ... or should we save those details for later? 16:22
kparal well 16:22
kparal there will be a Red Hat developer conference this weekend in Brno: https://fedoraproject.org/wiki/DeveloperConference2011 16:22
kparal I will be giving a short AutoQA workshop there 16:22
kparal I intend to show people how to create a really simple AutoQA test 16:23
kparal if I have any interesting and usable outputs of that workshop, I'll surely publish it 16:23
jlaska looks like lennart will be there ... any chance he has thoughts on some sort of systemd-lint tool (or something analogous to our initscripts tests)? 16:24
kparal jlaska: I can surely ask him 16:24
* Viking-Ice joins in late had a meeting conflict 16:24
jlaska kparal: thanks, if you have time to grab him for some input 16:24
kparal (but I should study systemd first, so I don't look too stupid when asking about it :) ) 16:24
jlaska Viking-Ice: welcome 16:25
jlaska kparal: I'd basically want to know if there is anything we can do to lint a package that includes a systemd script. Or how we should extend the initscripts tests to support systemd 16:25
jlaska (nothing concrete, just exploring ideas) 16:25
kparal jlaska: sure, great question 16:26
jlaska quite a lot of updates for the last 2 weeks ... anything else to note? 16:26
kparal that's all from me 16:26
jlaska kparal: sweet, thanks Kamil 16:26
jlaska a few reminders ... 16:27
jlaska #topic Tue, Feb 08 - Alpha 'Test Compose' 16:27
jlaska #info task#18 on Fedora 15 QA calendar (http://poelstra.fedorapeople.org/schedules/f-15/f-15-quality-tasks.html) 16:27
jlaska #undo 16:27
zodbot Removing item from minutes: <MeetBot.items.Info object at 0x2b02592df9d0> 16:27
jlaska #info task#19 on Fedora 15 QA calendar (http://poelstra.fedorapeople.org/schedules/f-15/f-15-quality-tasks.html) 16:27
jlaska there we go 16:27
Viking-Ice I think the biggest question here is anaconda status so far any livecd/usb installation is a no go 16:27
Viking-Ice will that work on alpha? 16:28
Viking-Ice installing that is 16:28
jlaska It should, but that's why we have these milestones to identify and prioritize the pain points 16:28
jlaska I'm not sure if rel-eng has filed a ticket yet for the Alpha deliverables ... I'll check into that after the meeting 16:28
* adamw notes we seem to have missed two alpha blocker meetings already 16:28
adamw whoops 16:28
brunowolff I have been looking at bug 672265. 16:29
adamw or, I mean...you guys didn't show up! 16:29
brunowolff It doesn't seem to be an alpha blocker. 16:29
Viking-Ice I did mentally not physically 16:29
Viking-Ice showing up that is 16:29
maxamillion adamw: for shame! 16:29
adamw if that's the liveinst fails bug and it's not marked as an alpha blocker, mark it 16:29
jlaska adamw: yeah ... that makes two of us :( 16:29
jlaska I don't see a ticket for this yet at https://fedorahosted.org/rel-eng/report/3 16:29
brunowolff I'd like to make it NTH, but I am not sure what is the right way to do that. 16:29
* rbergeron wonders if alpha blocker meetings are things she's supposed to be doing and completely failed 16:29
adamw rbergeron: we can all share the fail! 16:30
rbergeron adamw: GROUP HUG 16:30
jlaska rbergeron: we can chat after ... poelcat would drive a lot of these, and we can certaily use help balancing the load :) 16:30
rbergeron jlaska: sounds good 16:30
rbergeron i will be here :) 16:30
rbergeron (and elsewhere) 16:30
adamw sometimes poelcat drove, sometimes I did; I don't think it's officially part of your Job Description though, just something he did to spread the load 16:30
jlaska #action jlaska - ask rel-eng to file tickets for upcoming Alpha milestones 16:30
brunowolff If it really is a blocker, then the criteria should be changed. The install criteria only applies to install only images, 16:30
brunowolff according to the wiki. 16:31
brunowolff I'd like to get a dracut or dm expert to look at that bug. 16:31
adamw brunowolff: no, it doesn't 16:31
brunowolff Anyway I'll add it to the alpha blocker tracking bug. 16:31
adamw "The installer must boot (if appropriate) and run on all primary architectures from default live image, DVD, and boot.iso install media " 16:31
adamw is one of the Alpha criteria 16:31
maxamillion solid criteria 16:32
jlaska robatino are you available to assist rhe with wiki and announcement magic for the validation events? 16:32
adamw and then "The installer must be able to complete package installation with the default package set for each supported installation method " 16:32
brunowolff OK, I was looking through the detail breakout and didn't set it there. And in the breakout it specifically mentions 16:32
adamw yeah, basically, the alpha criteria imply that any complete fail bug in the installer is a blocker. 16:32
brunowolff install only images. That's a bit confusing. 16:32
jlaska I suspect we'll need a new anaconda build shortly ... I'll ping clumens 16:33
robatino jlaska: yes - i haven't been paying that much attention lately - is the procedure the same as before? 16:33
jlaska #action jlaska - ping clumens about a new anaconda build for the alpha TC 16:33
jlaska robatino: yeah should be 16:33
Viking-Ice jlaska: yeah his last one was from 25 jan I believe 16:33
adamw brunowolff: the only one that mentions them specifically is #3, and that's because live images don't boot to anaconda 16:33
jlaska okay ... so $topic will offer some excitement by way of bugs this week 16:33
jlaska any other questions on this topic? 16:33
adamw er, i mean, don't offer install options 16:33
jlaska robatino: if I haven't said it enough ... thanks for helping with the wiki/announce for these events :) 16:34
jlaska okay ... next up ... 16:34
adamw it does look like something's missing there, though, i'm sure there's supposed to be a criterion about the live image's boot menu 16:34
adamw will investigate later 16:34
jlaska #topic Thu, Feb 10 Test Day -- FreeIPA v2 16:34
* jlaska looks in TRAC to see who requested this event 16:34
brunowolff The bug is now a blocker for F15Alpha. 16:34
jlaska #link https://fedorahosted.org/fedora-qa/ticket/163 16:35
jlaska looks like this came from Dmitri Pal 16:35
jlaska anyone have time to reach out to dpal to see how test day prep is coming along? 16:35
jlaska if not ... I'm happy to ping 16:35
rbergeron btw: this is a feature that is going to FESCo on wednesday for feature approval - they forgot to move it to featurereadyforwrangler on the wiki. 16:36
Viking-Ice this one requires a bunch of docs + debug info 16:36
Viking-Ice as in we need to provide setup instruction for each service for the test day I believe 16:36
Viking-Ice and debug info would be good to have/finish writing that up 16:37
jlaska Viking-Ice: likely, I'd like to see what they wanted to accomplish for the test day 16:37
adamw i can get in touch with dmitri 16:37
Viking-Ice adamw: feel free to cc me to that 16:37
jlaska adamw: thanks 16:37
adamw Viking-Ice: roger 16:37
adamw can you #action it for me jlaska? 16:37
jlaska #action adamw - reach out to dmitri to check-in on FreeIPA test day prep 16:37
adamw yay 16:37
maxamillion <3 meetbot 16:38
jlaska we can always post-pone the event if sufficient time isn't available to properly setup this event 16:39
adamw yup 16:39
Viking-Ice agreed 16:39
jlaska anything else here ... 16:39
jlaska #topic Fri, Feb 11 - Alpha Blocker Meeting (f15alpha) #3 16:40
jlaska we kind of discussed this already, but just wanted to remind everyone about the Alpha blocker review scheduled for this Friday 16:40
adamw the first two went so well! 16:40
jlaska they certainly did! 16:40
jlaska EFAIL 16:40
jlaska adamw: rbergeron: Unless any objections, I'll grab announcing and meetbot duties for the event this Friday? 16:41
rbergeron works for me. I will watch and learn ;) 16:41
rbergeron or pitch in if someone throws something at me. 16:41
adamw sure 16:41
jlaska learn how *not* to host them :) 16:41
jlaska #link https://fedoraproject.org/wiki/QA:SOP_Blocker_Bug_Meeting 16:41
jlaska not sure there is anything else to add here 16:42
jlaska is there a BugZappers meeting this week? 16:42
jlaska do we need to remind folks about F15Alpha and the nice-to-have list? 16:42
adamw can't hurt 16:43
adamw the f15 Alpha blocker bug is https://bugzilla.redhat.com/show_bug.cgi?id=f15alpha 16:43
jlaska #action jlaska - send F15Alpha blocker review announcement 16:43
adamw if you think a bug should block the release of F15 Alpha, set it as blocking that bug, and it will be reviewed at the meeting 16:43
adamw the F15 Alpha nice-to-have bug is https://bugzilla.redhat.com/show_bug.cgi?id=f15alpha-accepted 16:43
jlaska #link https://bugzilla.redhat.com/show_bug.cgi?id=f15alpha 16:43
adamw that's for bugs which don't block the release but for which fixes should be allowed through alpha freeze 16:44
adamw you can propose bugs for it directly if you're sure they don't meet the blocker requirements; otherwise we usually put things on it when they don't get accepted as blockers 16:44
jlaska ... and fixing them isn't considered invasive to the release 16:45
jlaska s/invasive/disruptive/ 16:45
jlaska adamw: thanks for the links 16:45
jlaska next up, a topic kparal raised on the list ... 16:45
jlaska #topic Testing Fedora 15 in KVM virtual machines 16:45
adamw good one 16:45
kparal yes, there was a short discussion on the list 16:46
jlaska I believe Kamil wanted to note that using KVM guests no longer exercises the intended default desktop user experience 16:46
kparal yes, and I want to ask what whether our approach to KVM testing changed and how 16:46
jlaska #link http://lists.fedoraproject.org/pipermail/test/2011-February/096856.html 16:46
adamw we may have to tweak a few notes on criteria pages &c 16:46
jlaska perhaps, I suspect those will fall out during the blocker reviews? 16:47
adamw it's probably worth talking to virt team about acceleration passthrough plans 16:47
Viking-Ice this only applies to Gnome desktop btw 16:47
Viking-Ice testing 16:47
jlaska Viking-Ice: yes, thanks 16:47
adamw i believe something's queued up for spice 16:47
jlaska kparal: my take is I still plan to test with virtualization, but one needs to understand what is being tested since the virt environment may not meet the expected results for a given case 16:48
jlaska so testing with virt alone for the entire release isn't sufficient 16:48
kparal well, it's clear that we can't test default desktop in KVM. we can test just the fallback mode, and not to the full extent 16:48
adamw it'll be fine for exercising the live installer 16:48
Viking-Ice jlaska: *cough* it never is 16:48
adamw it'll be useless for the desktop validation testing 16:48
jlaska Viking-Ice: *exactly* 16:49
kparal I believe we can still do installation testing, but we must be clear what "system boot successfuly" means 16:49
maxamillion adamw: well, that's true 16:49
adamw as things stand all gnome desktop validation will have to be done on raw metal 16:49
vhumpa kparal: Looks like at that point we'll simply have to use bare hardware 16:49
jlaska how does the fallback mode different on bare-metal vs virt? 16:49
maxamillion adamw: isn't bare metal the prefered case for all desktop validation? just so we can test with different hardware drivers and such? 16:49
adamw kparal: that criterion is really intended for the _installer_ per se - does the installer properly render the installed system bootable 16:50
adamw kparal: bugs in the actual desktop installed are meant to be covered by other criteria 16:50
adamw maxamillion: sure, doing it in virt can be handy for some testing though 16:50
adamw desktop menus koff koff 16:50
kparal jlaska: quoting JBG: For any Gnome related test days you cant use KVM since one of the 16:50
kparal information we need to gather from test days for developers is that if 16:50
kparal users are getting either a shell or fallback mode or not and those 16:50
kparal reporters that aren't getting shell or fallback mode will need to file a 16:50
kparal bug providing their smolt profile so developers can look deeper into 16:50
kparal what hw is failing to reach shell or fallback mode. 16:50
jlaska maxamillion: adamw: virt is another environment, just like bare-metal 16:50
maxamillion jlaska: I imagine it will be the same if it works, but likely if everyone is testing on virt and there's some issue with the hand off to fall back mode based on some specific graphics card (or $other factor) it would be missed with virt 16:51
jlaska maxamillion: definitely 16:51
Viking-Ice yup any Gnome testing needs to be performed on bare metal 16:51
maxamillion adamw: right, I do all my pre-alpha rawhide testing in virt because I don't have enough spare hardware to have rawhide eating my hard drives contents 16:52
maxamillion jlaska: +1 16:52
jlaska kparal: hmm ... I'll need to confirm with jrb, since I think virt is still valid for testing fallback support ... but *not* for testing the default GNOME experience 16:52
adamw it 16:52
adamw it's valid for testing the fallback experience 16:52
kparal jlaska: I believe that too 16:52
adamw as noted, we do need bare metal testing to make sure the fallback _mechanisms_ work 16:52
Viking-Ice yup which is pretty much what I said in that thread 16:52
jlaska indeed 16:52
adamw kvm is just one fallback case out of...zillions 16:52
jlaska well said 16:53
kparal ok, we all understand then :) 16:53
Viking-Ice :) 16:53
vhumpa no problem for me to test on raw hw, as soon as the rawhide is properly installable 16:53
kparal vhumpa: good point :) 16:53
adamw heh 16:53
jlaska proposed - #agreed Using KVM for testing is still supported and needed. However, recognize that the virtual environment is just one of *many* environments that needs testing, and doesn't offer testing ofthe default GNOME3 user experience 16:53
maxamillion Xfce Spin works in full featured mode on KVM! </shameless plug> 16:54
jlaska +1, -1 16:54
kparal should we alter the desktop validation matrices? 16:54
maxamillion >.> 16:54
kparal gnome-shell vs fallback mode? 16:54
Viking-Ice vhumpa yeah I'm experiencing bugs on my update-to-rawhide that I did not see on the compose ( live usb ) that our newly crowned insomnia king compose for Gnome test day:) 16:54
maxamillion kparal: might be a good idea 16:54
adamw kparal: yeah, proposals welcome 16:55
jlaska presently, we leave it open as an exercise for the user 16:55
adamw note that i wrote a metric assload of new test cases for the gnome test day, any of which we can slot in as validation tests if it seems appropriate 16:55
jlaska do we want to exhaustively list all envirements, or just bare-metal and virt, or list several fall-back tests 16:55
adamw we could just have an extra column - one for gnome shell, one for fallback 16:55
vhumpa viking-ice: yes, we can't have somebody loose all sleep over life cd's :) 16:56
jlaska adamw: yes! 16:56
adamw but yeah, let's not hash it out here 16:56
jlaska agreed 16:56
adamw on-list for proposals? 16:56
jlaska #agreed Using KVM for testing is still supported and needed. However, recognize that the virtual environment is just one of *many* environments that needs testing, and doesn't offer testing of the default GNOME3 user experience 16:56
jlaska #info Adjustments to the desktop validation matrix (https://fedoraproject.org/wiki/QA:Desktop_validation_testing) may be required to accomodate for fallback mode, and/or other hardware environments 16:57
jlaska at a mimimum, I think'll need a GNOME specific fallback test 16:57
jlaska okay ... we are approaching the hour mark 16:57
jlaska time for open discussion ... 16:57
jlaska #topic Open discussion - <Your topic here> 16:57
adamw so, we had a lil' test day last week you may have heard of 16:58
maxamillion adamw: +1 to on-list for proposals 16:58
jlaska adamw: did you want to discuss that here? 16:59
adamw i was gonna do a wrap-up 16:59
Viking-Ice I got one.. I think we should make it a requirement for test days to have how_to_debug pages ready present for the components that are being tested present and ready before the test day is held :) 16:59
jlaska #topic GNOME3 Test Day wrap-up 16:59
adamw #link https://fedoraproject.org/wiki/Test_Day:2011-02-03_GNOME3_Alpha 16:59
jlaska Viking-Ice: that seems like a nice-to-have to me ... it's not entirely easy to create those pages, in addition to the other tests and wiki pages required for the test day 17:00
adamw we had the first of three GNOME 3 test days on thursday, it went very well thanks to heroic work by the desktop team to get testing packages ready 17:00
adamw *waves big #topic stick* 17:00
* jlaska remains quiet until #topic has concluded 17:00
adamw we had several dozen testers and lots of juicy bugs reported 17:00
adamw thanks to everyone who came and tested 17:00
jlaska +1 17:01
adamw i'll do a proper wrap-up mail soon 17:01
* rbergeron has a question? 17:01
adamw question away! 17:01
jlaska sweet, thanks Adam. I wasn't able to participate during the event ... but the wiki is quite active 17:01
rbergeron possibly noob question. even very likely. 17:01
adamw set phasers to laugh 17:02
rbergeron So all of the bugs found were getting reported against GNOME's bz, is that correct? 17:02
adamw most of them 17:02
Viking-Ice really I filed mine against rh bugzilla 17:02
adamw things that are clearly fedora packaging issues went to RH bugzilla 17:02
vhumpa adamw: thanks for the day, I think that was particularly nice test-day for a newbie 17:02
adamw Viking-Ice: the notes said to use GNOME bugzilla, but never mind :) it's not hugely important 17:02
adamw vhumpa: glad it was good for you@ 17:02
rbergeron I guess I'm confused as to how we track those back to Fedora to see if things are fixed. Is it just "we hope testing goes better next time" or do we actually keep track of the bugs we filed over there or ... ? 17:03
rbergeron or not that testing goes better, since it went pretty well, 17:03
jlaska were any of the upstream bugs added to the shell blocker list? 17:03
rbergeron but we hope that next round of testing yields less bugs. 17:03
Viking-Ice I personally am against letting reporters run around the half the internet filing in each and every bugzilla track instance out there 17:03
adamw haven't done all the tracking yet 17:03
adamw Viking-Ice: this was what the developers requested, when we asked 17:03
* rbergeron is'nt harassing, just curious about processes i'm unfamiliar with thus far :) 17:03
adamw we checked with j5, caillon, and mclasen 17:03
adamw rbergeron: we have a very dumb little script i hacked up which runs over the test day page and pulls out any bug URLs 17:04
adamw rbergeron: i'll have to tweak it slightly to look for gnome as well as RH bz for this day 17:04
jlaska Viking-Ice: I don't think we asked them to file where ever they wanted ... it was requested that upstream issues be reported upstream, and packaging be reported against Fedora. 17:04
vhumpa yes, i think i also filled a bug with RH, that should have been to Gnome 17:04
adamw rbergeron: you can fire off that script whenever you like and track the changes to the reported bugs 17:04
Viking-Ice I think we need to get a larger reporters feel about running having upstream accounts for components we test against 17:05
rbergeron adamw: that sounds FANCY. :) 17:05
adamw rbergeron: for X test days, I've usually reviewed how bugs reported were handled later in the cycle, check the archives for some of those mails; we can do the same for GNOME and probably will 17:05
Viking-Ice s/running/ 17:05
Viking-Ice if maintainers request that we should reject it is my take on it 17:05
adamw it didn't seem to cause any problems. 17:05
jlaska Viking-Ice: hrmm, probably something we can hash outside of meeting ... but I don't see a reason to mandate that upstream bugs should be filed downstream 17:06
adamw rbergeron: the 'script' is at the bottom of https://fedoraproject.org/wiki/QA/SOP_Test_Day_management , it's really not fancy at all 17:06
rbergeron adamw: i shall check it out :) 17:06
tflink adamw: do you know how many bugs were filed in RH bugzilla that should have been filed with GNOME? 17:06
adamw it just pipes the test day page through a hideous heath robinson arrangement of text processing tools :P 17:06
rbergeron anyway. SORRY FOR NOOB DISTRACTION :) 17:06
adamw tflink: nope. again, i haven't been through all the post-event teardown yet 17:06
adamw i needed to do dumb things like sleep and learn to snowboard 17:06
jlaska details 17:06
jlaska alright ... let's start wrapping up 17:07
adamw i'll probably do it today. 17:07
jlaska anything else on GNOME3 test day? 17:07
Viking-Ice jlaska: we are downstream and requiring that reporters have upstream bugzilla account and equivalent does not scale. one account in our bugzilla should suffice 17:07
jlaska adamw: roger, thanks 17:07
adamw Viking-Ice: it's not something i'd expect to happen a lot. 17:08
jlaska Viking-Ice: mandates like that don't scale imo ... test days are designed to be flexible to meet the needs of maintainers and packagers 17:08
adamw the gnome events are somewhat special as they're really combined gnome3 / fedora 15 test days, and were billed as such. 17:08
jlaska if having an upstream bugzilla account was a big blocker, we could easily just file tickets to bugzilla.redhat.com and later migrate them (by hand or script) 17:08
jlaska but it doesn't sound like it was a tremendous blocker to participation 17:08
jlaska #topic Open Discussion - <your topic here> 17:09
adamw yeah, we didn't get any questions/complaints about it during the day. 17:09
Viking-Ice jlaska: now comes KDE test day and reporters need to have KDE upstream account etc.. 17:09
Viking-Ice pandora box opening 17:09
Viking-Ice you allow one we have to allow them all 17:09
jlaska perhaps, we'll deal with that when we get there 17:09
vhumpa Loking forward to that :) 17:09
Viking-Ice or take preventing measures 17:09
jlaska we allow what maintainers ask for 17:09
Viking-Ice why are we then using our bugzilla et al 17:10
tflink adamw: I can't speak for anyone else, but I completely missed the upstream account requirement. That may have affected feedback 17:10
jlaska Viking-Ice: off-topic ... I don't think we really need to answer that, but I'll be happy to on #fedora-qa 17:10
adamw tflink: yeah, some bugs may be in RH bugzilla that probably shouldn't be. again, it's not a huge deal breaker or anything. 17:10
adamw (practically speaking, the owners of the bugs will be the same people half the time anyway.) 17:11
Viking-Ice jlaska: why off topic 17:11
Viking-Ice ? 17:11
jlaska Viking-Ice: discussing why we have a bugzilla.redhat.com is not relevant to this discussion 17:11
Viking-Ice ok heres a topic for the feeding should we or should we not allow maintainers to require direct upstream filing and accounts for test days 17:12
Viking-Ice you can set that as info gather feed back from test list 17:12
jlaska #info should we or should we not allow maintainers to require direct upstream filing and accounts for test days 17:12
jlaska to me, that feels like a distraction to debate over 17:13
Viking-Ice ? 17:13
adamw yeah 17:13
Viking-Ice distraction of what 17:13
Viking-Ice exactly 17:13
adamw we can't enforce a requirement like that, anyway 17:13
Viking-Ice yes we can we can drop such request from maintainers 17:14
adamw it just doesn't feel like something particularly productive to worry about, to me 17:14
Viking-Ice and require them to actually use our bugzilla instance 17:14
jlaska Viking-Ice: it's not clear what we lose or gain by investing in that requirement 17:14
adamw we can add a note to the SOP explaining that an external bugzilla request is a barrier to entry and to avoid it if possible 17:14
jlaska it seems like "process" for the sake of process 17:14
jlaska was it a barrier? 17:14
adamw but phrasing things as requirements and Thou Shalt Nots just doesn't feel terribly helpful to me 17:14
jlaska bugs got filed 17:14
adamw anyone else have an opinion? 17:14
adamw jlaska: in theory it is, and it's hard to prove that it's not 17:14
Viking-Ice I want this topic to reach the wider audience please 17:14
jlaska the preference was to file upstream, worst case ... they were filed in bugzilla.redhat.com 17:15
adamw jlaska: we do get reports where people just don't file bugs, and as tflink said, some people seem to have filed in RH bugzilla not upstream 17:15
jlaska Viking-Ice: okay, do you want to take this to list@ please? 17:15
adamw so it's hard to say definitively 17:15
Viking-Ice jlaska: sure throw that task at me 17:15
jlaska I'm open to being wrong, it happens a lot :) 17:15
* kparal votes to solve this on test list 17:15
adamw check the last (currently) entry on the page, for e.g. - lots of notes about interesting-looking bugs, no reports filed 17:15
* kparal can't respond, you're typing too fast :) 17:15
jlaska this just feels like another bike shed discussion 17:15
* rbergeron hands jlaska the blue paint 17:15
jlaska I'd definitely want feedback from those who have organized test days, or maintainers who have pitched test days 17:16
adamw yeah, i feel bike shed-y. 17:16
jlaska #action Viking-Ice - solicit feedback on test@lists.fedoraproject.org to see whether we need to require only bugzilla.redhat.com use during test days 17:16
Viking-Ice jlaska: this does not scale requiring or requesting that reporters file upstream bugs reacquires them to have in most case upstream account 17:17
adamw i think viking's other suggestion was a lot more interesting, though again, shouldn't be phrased as a requirement. 17:17
jlaska suggestion, yeah 17:17
* jlaska missing the scaling problem 17:17
adamw me too 17:17
jlaska what doesn't scale to me is having only a few people pitch and host test days 17:17
jlaska not which bug tracker to use 17:17
adamw it's not like we have people showing up to every single test day and bugzilla accounts take up 1GB of hard disk space each 17:17
adamw i have hundreds of the things 17:17
adamw took me two minutes each to sign up for 17:18
Viking-Ice I think I have about 25 upstream accounts already only for the purpose to file upstream against various components <sigh> 17:18
jlaska when developers/maintainers complain that they are having a hard time tracking bugs from the test days ... then we get dig deeper 17:18
jlaska alright ... the fuse has been set for 1 minute 17:18
adamw it's a very small added requirement for any one particular test day's testers, that's about all 17:18
rbergeron if you're interested in signing up for an hour or two of help, the extra 2 minutes to get an upstream bz account shouldn't hurt. what hurts is when upstream doesn't know all of the possible places to check, where duplicate bugs are reported, and things just wind up not getting fixed, would be my guess. 17:18
jlaska yes, well phrased 17:19
tflink is a noob, but votes for whatever is easiest for testers and people running the test days - more participation == better 17:19
Viking-Ice it's the maintainers/packager job to keep tab on that stuff 17:19
jlaska 30 seconds ... 17:19
rbergeron i think we have a vested interest in helping to make sure that GNOME goes as smoothly as possible out the door. 17:19
adamw okay, now you're getting into the wider philosophical argument about whether users should file bugs downstream and maintainers move them upstream, which I REALLY don't want to get into here 17:19
jlaska bingo 17:19
rbergeron if that means we ask people to kindly file things upstream, then so be it. :) 17:19
adamw it goes around all the time on the lists and never goes anywhere and just wastes everyone's time 17:20
adamw so i'm not gonna engage with that discussion... 17:20
jlaska new topic ... or end of meeting gang 17:20
rbergeron particularly if it makes their lives # 17:20
rbergeron # Features/LZMA for Live Images 17:20
rbergeron oh, that's what i'm working on. 17:20
rbergeron ... easier. :) 17:20
jlaska this is the longest 2 minutes ever 17:20
jlaska 10 seconds until #endmeeting 17:20
* rbergeron hands jlaska a timer 17:20
jlaska Alright, thanks everyone for input+discussion+debate 17:21
jlaska let's continue on the list as needed 17:21
jlaska I'll follow-up to the list with minutes, later today 17:21
jlaska #endmeeting 17:21

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!

Retrieved from "https://fedoraproject.org/w/index.php?title=QA/Meetings/20110207&oldid=219932"