QA/Meetings/20090701

From FedoraProject

Jump to: navigation, search

Contents

Attendees

  • Jóhann Guðmundsson (viking-ice)
  • Will Woods (wwoods)
  • Matthias Clasen (mclasen)
  • David Pravec (dpravec)
  • James Laska (jlaska)
  • Southern_Gentleman
  • Seth Vidal (skvidal)

Regrets:

  • Adam Williamson (adamw)
  • Jesse Keating (f13)


Agenda

Proposed meeting agenda

Previous meeting follow-up

  • [jlaska] - update QA/Goals wiki document

I don't have big updates here, other than I finally dusted off my fingers and am getting started. I should have draft content soon. One of the pre-reqs for me was pulling together the test day feedback ... so having that done helps me now (https://www.redhat.com/archives/fedora-test-list/2009-July/msg00009.html).

  • [viking_ice] - check-in with abrt

Viking_ice had to step away briefly. Looked into the abrt project over the week. Filed a RFE with several mock-ups designed to improve the bugzilla integration (see https://fedorahosted.org/abrt/attachment/ticket/53/Automatic%20Bug%20Reporting%20Tool%20Mockup1.png and https://fedorahosted.org/abrt/attachment/ticket/53/ABRT_simple_bug_filing_form.png)

  • [wwoods] - add post-install network test to basic functionality in rawhide acceptance plan

I added two cases to the plan, post-install network and post-install input. For additional information, please see https://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan#Basic_Functionality.

Fedora 12 Fit n Finish Test Days

Announcement: https://www.redhat.com/archives/fedora-test-list/2009-June/msg00741.html

mclasen briefed the team on the upcoming desktop-focused Fit_and_Finish test days. Traditional fedora test days focus on exercising upcoming features of the release. The goal of Fit'n'finish is to do a series of user-task focused test days, that are delivered with in the same manner as traditional Fedora test days. The first event will be this Tuesday (July 07, 2009) with a focus on Fit_and_Finish/Display_Configuration.

Southern_Gentleman asked if rawhide would be used for generating the live images? Mclasen indicated the plan is to use rawhide, but I don't think it is essential for user-task testing that everyone runs the same.

Viking_ice indicated that there aren't clear instructions on what to include in bug reports filed. Mclasen indicated he would add some clarity here after responding to the mail thread. Jlaska recommended linking to Category:Debugging and that the QA team could help fill information gaps.

AutoQA - update from wwoods

https://fedorahosted.org/autoqa

The QA:Rawhide_Acceptance_Test_Plan is complete. 7 out of 12 test cases created from the QA:Rawhide_Acceptance_Test_Plan.

The plan for next week:

  • have the test cases all finished
  • at least one automated

jlaska asked if there were any good candidates for tests that could be rapidly developed ... something off-the-shelf. Wwoods indicated there are a few good candidates, including:

  • the repodata/comps validity stuff
  • installer image existence

Wwoods indicated the need for reporting manual test results. Jlaska asked if this was intended as a generic test result reporting mechanism? Wwoods outlined he is trying to solve a specific problem of reporting test results for rawhide acceptance tests not yet automated, not rewrite a formal test case management system from scratch.

Install Testing SOP draft - update from lili

https://www.redhat.com/archives/fedora-test-list/2009-June/msg00734.html

Hopefully Liam is sleeping right now (being 12:44am for him) ... but wanted to give a quick update on an SOP he posted for review. Liam is trying to documenting the tasks involved when planning+preparing and executing a collaborative installation test event.

A few folks have chymed in already and helped clean up the wiki, so thanks to arxs and beland! While this largely affects the folks who have been testing RC's, feedback from a wider audience is encouraged ... as that's the audience we're hoping benefits from added transparency

Improve Bug Reporting proposal - update from viking_ice

https://fedoraproject.org/wiki/User:Johannbg/QA/Improve_reporting

Viking_ice discussed several explorations with the abrt tool and discussed several ideas on improving the user-interface. This included several mock-ups posted to the abrt trac instance (https://fedorahosted.org/abrt/attachment/ticket/53/Automatic%20Bug%20Reporting%20Tool%20Mockup1.png and https://fedorahosted.org/abrt/attachment/ticket/53/ABRT_simple_bug_filing_form.png).

Viking_ice suggested that abrt could be better if it were very similar to the simplified bugzilla reporting UI. This may reduce the learning curve

Mclasen suggested that several iterations would be needed to clean up the UI, and that assistance from the design team would certainly help.

Viking_ice also pointed to other efforts that improved the UI of setroubleshoot (see Design/SETroubleshootUsabilityImprovements). Specifically, the screenshots of the Report Bug screen.

Open discussion

Fedora Test Day Survey

Jlaska finally gathered F11 test day survey feedback and started posting recommendations for review. Please see https://www.redhat.com/archives/fedora-test-list/2009-July/msg00009.html

Upcoming QA events

Action items

  • [jlaska] - update QA/Goals wiki document

IRC Transcript

jlaska | #startmeeting Jul 01 12:00
fedbot | Meeting started Wed Jul 1 16:00:07 2009 UTC. The chair is jlaska. Jul 01 12:00
fedbot | Information about MeetBot at http://wiki.debian.org/MeetBot , Useful Commands: #action #agreed #halp #info #idea #link #topic. Jul 01 12:00
jlaska | #meetingtopic Fedora QA meeting Jul 01 12:00
--- | fedbot has changed the topic to: (Meeting topic: Fedora QA meeting) Jul 01 12:00
jlaska | #topic gathering Jul 01 12:00
--- | fedbot has changed the topic to: gathering (Meeting topic: Fedora QA meeting) Jul 01 12:00
jlaska | Hey gang, show of hands folks attending the Fedora Quality Assurance meeting Jul 01 12:00
jlaska | I know adamw and f13 are unable to attend today Jul 01 12:01
Viking-Ice | hellú.. Jul 01 12:01
* | wwoods waves Jul 01 12:01
jlaska | Viking-Ice: wwoods: Hey guys! Jul 01 12:01
* | mclasen pays a visit Jul 01 12:01
jlaska | mclasen: welcome :) Jul 01 12:01
dpravec | hello all, i am new here, i have joined fedora qa in Brno today Jul 01 12:02
jlaska | dpravec: glad to have you, thanks for joining in from home! Jul 01 12:03
jlaska | okay, I think that captures most folks ... let's dive into the brief follow-up items from last week Jul 01 12:04
jlaska | #topic Previous meeting follow-up Jul 01 12:04
* | skvidal is lurking Jul 01 12:04
--- | fedbot has changed the topic to: Previous meeting follow-up (Meeting topic: Fedora QA meeting) Jul 01 12:04
jlaska | skvidal: hey there Jul 01 12:04
jlaska | Okay, I've got the still outstanding item of updating our goals page to be more current Jul 01 12:05
jlaska | * [jlaska] - update QA/Goals wiki document Jul 01 12:05
jlaska | I don't have big updates here, other than I finally dusted off my fingers and am getting started. I should have draft content soon. Jul 01 12:05
jlaska | One of the pre-reqs for me was pulling together the test day feedback ... so having that done helps me now Jul 01 12:05
jlaska | as always, input welcome on the test day survey or recommendations ... https://www.redhat.com/archives/fedora-test-list/2009-July/msg00009.html Jul 01 12:06
jlaska | s/welcome/encouraged/ Jul 01 12:06
jlaska | # [viking_ice] - check-in with abrt Jul 01 12:06
jlaska | Viking-Ice: I've got you down next for a update on some discovery efforts around abrt? Jul 01 12:06
jlaska | uh oh, we may have lost Viking-Ice Jul 01 12:08
jlaska | okay, we can come back to this. Jul 01 12:10
jlaska | I spoke to Viking-Ice earlier today on the abrt subject, so I believe he's been poking around in that space and offering some RFE's for the abrt team. Jul 01 12:10
jlaska | * [wwoods] - add post-install network test to basic functionality in rawhide acceptance plan Jul 01 12:10
jlaska | I believe you had this done before last weeks meeting concluded? Jul 01 12:11
jlaska | wwoods: did you go where Viking-Ice went? Jul 01 12:13
wwoods | yeah, I added two cases to the plan Jul 01 12:13
wwoods | post-install network and post-install input, I think Jul 01 12:13
jlaska | Nice, what's the linky? just https://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan#Basic_Functionality Jul 01 12:14
jlaska | okay ... going with that for now Jul 01 12:15
jlaska | #topic F12 - Raising the bar Jul 01 12:15
--- | fedbot has changed the topic to: F12 - Raising the bar (Meeting topic: Fedora QA meeting) Jul 01 12:15
jlaska | we're missing a few folks today, so much of the agenda is just touching in on several fronts Jul 01 12:15
jlaska | The first is an effort mclasen announced to the list ... see https://www.redhat.com/archives/fedora-test-list/2009-June/msg00741.html Jul 01 12:16
mclasen | not sure if I need to say something about it ? Jul 01 12:16
mclasen | we want to do a series of user-task focussed test days Jul 01 12:16
mclasen | rather than feature-focussed Jul 01 12:17
mclasen | I hope some of the fine qa folks come by... Jul 01 12:17
mclasen | We want to do the first one next Tuesday Jul 01 12:17
jlaska | there's been some interesting feedback on the f-d-list for this, but overall this is a really cool initiative. I'm looking forward to contributing! Jul 01 12:18
Southern_Ge | ? Jul 01 12:18
jlaska | mclasen: is the schedule of events - http://fedoraproject.org/wiki/Fit_and_Finish ? Jul 01 12:18
mclasen | yes, as far as we have it yet Jul 01 12:18
mclasen | I expect this effort to start a bit slow, and then pick up speed Jul 01 12:19
jlaska | mclasen: are there any additional preparations you guys are looking to make for the event? Jul 01 12:19
mclasen | as more of my team finish their f12 feature work and can join in Jul 01 12:19
mclasen | jlaska: we are trying to get live cds building and working Jul 01 12:19
mclasen | ajax was looking into that earlier today Jul 01 12:19
mclasen | so far we have the first (ie have them building) Jul 01 12:19
mclasen | but not the second (boots into runlevel 3, but no X) Jul 01 12:20
mclasen | also, they're a bit oversize Jul 01 12:20
jlaska | mclasen: you guys are closer to the issues affecting building them on rawhide right now, but if we can help ... just shout Jul 01 12:20
* | jlaska hasn't tried building one this month ... kind of a nice break from that :) Jul 01 12:20
mclasen | I expect ajax will maybe post an update on that sometime soon Jul 01 12:20
mclasen | I'll ask him Jul 01 12:20
mclasen | even better, he's here... Jul 01 12:20
mclasen | other than that, for preparations, I will spend some more time filling up the test day page with information Jul 01 12:21
mclasen | and I have to figure out how we are going to handle the issues that are found Jul 01 12:21
jlaska | mclasen: you've got some nice focus areas listed http://fedoraproject.org/wiki/Fit_and_Finish/Display_Configuration Jul 01 12:22
mclasen | bugs with a tracker bug, I guess... Jul 01 12:22
jlaska | that's right Jul 01 12:22
Southern_Ge | mclasen, so you are using rawhide or f11 for your test days? Jul 01 12:22
jlaska | so for the QA folks, I encourage folks to join in. Our teams are working together so that we can provide a consistent test day experience ... while also helping to identify use case gaps + bugs Jul 01 12:23
mclasen | Southern_Gentleman: the plan is to use rawhide, but I don't think it is essential for user-task testing that everyone runs the same Jul 01 12:24
mclasen | you may well find issues in F11 that have a high chance to still be relevant in rawhide Jul 01 12:24
jlaska | as we move further into F12 development cycle, I suspect the gap will become wider? Jul 01 12:24
mclasen | hopefully we'll fix some of the issues that are found, so yes... Jul 01 12:25
jlaska | cool Jul 01 12:26
* | Viking-Ice back Jul 01 12:27
jlaska | Thanks for kicking things off on the Test Day front. Having you and your crew involved should really help fast track issues that come out of this Jul 01 12:27
jlaska | I've updated the upcoming events section to point to #fedora-fit-and-finish next Tuesday Jul 01 12:27
Viking-Ice | btw mclasen reporters are missing what to report on you Fit and finish Jul 01 12:27
Viking-Ice | which files to include etc .. Jul 01 12:28
mclasen | I will try to clarify some of that when I get to responding in that thread Jul 01 12:28
jlaska | mclasen: Viking-Ice raises a good point. Something that surfaced from last weeks meeting was the need for improved details on writing good bug reports Jul 01 12:28
jlaska | mclasen: if it makes sense at all to link to upstream docs or add some content into https://fedoraproject.org/wiki/Category:Debugging ... that's an area we can assist in Jul 01 12:29
Viking-Ice | mclasen: all X components are covered Jul 01 12:29
Viking-Ice | https://fedoraproject.org/wiki/Xorg/Debugging#All_bug_reports Jul 01 12:29
jlaska | okay ... going to move on to next topic (there is a meeting room conflict in 30 minutes) Jul 01 12:30
jlaska | #topic AutoQA - update from wwoods Jul 01 12:31
--- | fedbot has changed the topic to: AutoQA - update from wwoods (Meeting topic: Fedora QA meeting) Jul 01 12:31
jlaska | wwoods: what's the word on the autoqa front? Jul 01 12:31
wwoods | So the plan (as mentioned earlier) is complete: https://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan Jul 01 12:31
wwoods | 7/12 cases have been written Jul 01 12:32
wwoods | looking over the roadmap: https://fedorahosted.org/autoqa/milestone/israwhidebroken.com Jul 01 12:32
wwoods | the stuff there isn't really in order but all the steps are there. Jul 01 12:33
wwoods | so the next step after finishing writing test cases is to attempt to automate one Jul 01 12:33
wwoods | which requires an autotest instance, which requires packaging autotest Jul 01 12:33
wwoods | that's https://fedorahosted.org/autoqa/ticket/9 Jul 01 12:33
wwoods | jlaska and f13 are handling that and it seems to be going well Jul 01 12:34
jlaska | yeah ... good progress on that front Jul 01 12:34
jlaska | I'm happy with the autotest packaging, and have started futzing with the autoqa package to tie into it Jul 01 12:34
wwoods | hopefully by next week we'll have the test cases all finished and at least one automated Jul 01 12:34
wwoods | once we know how to write tests for autotest Jul 01 12:34
jlaska | I have to discuss with f13 how we'll manage some of the python dependencies to get this into shape for fedora-infrastructure Jul 01 12:34
wwoods | we can start trying to figure out how to organize and package those tests in the autoqa repo Jul 01 12:35
jlaska | wwoods: was there a test you had in mind that seemed the most ready for automating? Jul 01 12:35
jlaska | something where we can use the off-the-shelf approach? Jul 01 12:36
wwoods | the repodata/comps validity stuff is really easy to automate Jul 01 12:36
wwoods | as is installer image existence Jul 01 12:36
wwoods | there are existing scripts / tools / example code to mess with that stuff Jul 01 12:36
jlaska | cool, that should help! Jul 01 12:37
jlaska | I've got some bits of that .treeinfo validation if it's useful Jul 01 12:37
wwoods | the other thing to figure out is.. how to allow manual test reporting Jul 01 12:37
wwoods | since there's always going to be things that we can't automate (or haven't figured out how to automate yet) Jul 01 12:37
jlaska | wwoods: is this a general test result reporting solution? Jul 01 12:38
wwoods | that's a whole big can of worms, because then you need identification/authentication to know who is allowed to report the results Jul 01 12:38
wwoods | I really don't want to rewrite something like testopia Jul 01 12:38
wwoods | and we've been promised some magical Test Case Management System for months and months and it never seems to happen Jul 01 12:39
wwoods | because it's a huuuge problem to solve Jul 01 12:39
wwoods | so no - that would get way, way out of hand Jul 01 12:39
jlaska | agreed! Jul 01 12:39
wwoods | I'm trying to keep this as simple as possible Jul 01 12:39
jlaska | I don't know how robust it is, but I think autotest does support remote communication Jul 01 12:40
jlaska | so that might help solve the immediate need, who knows Jul 01 12:40
--- | warren__ is now known as warren Jul 01 12:41
jlaska | wwoods: thanks for the update, we'll stay tuned to fedora-test-list for any updates or if test review is needed Jul 01 12:42
jlaska | wwoods: any other updates on the autoqa front? Jul 01 12:43
wwoods | nothing I can think of at this time Jul 01 12:44
jlaska | okay, changing topic ... Jul 01 12:44
jlaska | #topic Install Testing SOP draft - update from lili Jul 01 12:44
--- | fedbot has changed the topic to: Install Testing SOP draft - update from lili (Meeting topic: Fedora QA meeting) Jul 01 12:44
jlaska | hopefully Liam is sleeping right now (being 12:44am for him) ... but wanted to give a quick update on an SOP he posted for review Jul 01 12:44
jeremy | jlaska: this is fedora, we don't allow sleeping :-P Jul 01 12:45
jlaska | Liam is trying to document the tasks involved when planning+preparing and executing a collaborative installation test event (see mail at - https://www.redhat.com/archives/fedora-test-list/2009-June/msg00734.html) Jul 01 12:45
jlaska | jeremy: shhh! aren't you supposed to be sleeping! Jul 01 12:45
jlaska | :) Jul 01 12:45
--- | warren is now known as warren_ Jul 01 12:46
jlaska | A few folks have chymed in already and helped clean up the wiki, so thanks to arxs and beland Jul 01 12:46
--- | warren_ is now known as warren Jul 01 12:47
jlaska | while this largely affects the folks who have been testing RC's, feedback from a wider audience is encouraged ... as that's the audience we're hoping benefits from added transparency Jul 01 12:47
jlaska | https://fedoraproject.org/wiki/User:Liam/Draft_Install_Test_SOP Jul 01 12:47
--- | denise_afk is now known as denise Jul 01 12:47
jlaska | unless any questions/comments ... changing topics ... Jul 01 12:48
jlaska | #topic Improve Bug Reporting proposal - update from viking_ice Jul 01 12:48
--- | fedbot has changed the topic to: Improve Bug Reporting proposal - update from viking_ice (Meeting topic: Fedora QA meeting) Jul 01 12:48
jlaska | Viking-Ice: wanted to give you an opportunity for any updates on the proposal you drafted Jul 01 12:48
jlaska | looks like some good stuff come from that discussion last week Jul 01 12:49
Viking-Ice | Yup things are emerging.. Jul 01 12:49
Viking-Ice | Took a look at abrt Jul 01 12:49
Viking-Ice | Filed an rfe for gui improvements https://fedorahosted.org/abrt/attachment/ticket/53/Automatic%20Bug%20Reporting%20Tool%20Mockup1.png and https://fedorahosted.org/abrt/attachment/ticket/53/ABRT_simple_bug_filing_form.png Jul 01 12:50
jlaska | Viking-Ice: I know the abrt guys follow their trac instance, but you may want to check in with them directly. I know they are keenly interested in improving the look'n'feel (and ease of use) of abrt Jul 01 12:51
* | mclasen recommends iterating over that design a bit  :-) Jul 01 12:51
jlaska | mclasen: you're much closer than me to what is helpful when it comes to designing GUI apps Jul 01 12:52
Viking-Ice | Bringing the gui closer to the simpler bug reporting from that's coming with the upcoming release of bugzilla The Idea here is to reduce the learning curve for reporters so if users have gotten used to work in the upcoming bugzilla simple reporting form they have no problem getting used to use abrt and also if we need to get reporters for some reason to bugzilla they would be familiar with the reporting form there... Jul 01 12:52
Viking-Ice | mclasen: over my proposed designed or the current? Jul 01 12:53
jlaska | I guess it's a question for the abrt team whether abrt is intended as a graphical desktop client for filing bugs, or a generic problem reporting tool ... where one of the reporting "plugins" is bugzilla? Jul 01 12:53
mclasen | Viking-Ice: both Jul 01 12:54
mclasen | the abrt guys need to get together with some of our designers, really Jul 01 12:54
jlaska | definitely! Jul 01 12:54
jlaska | Look at what huge improvements they've suggested for setroubleshoot ... https://fedoraproject.org/wiki/Design/SETroubleshootUsabilityImprovements Jul 01 12:55
Viking-Ice | Sorry to say but I think they should try to have this as close to the bugzilla form as possible.. Jul 01 12:55
* | mclasen doesn't even think bugzilla should figure at all in this... Jul 01 12:55
mclasen | anyway, probably offtopic here Jul 01 12:55
Viking-Ice | true Jul 01 12:56
jlaska | Viking-Ice: the Debugging pages were a nice offshoot from your topic last week, I know Adam is raising these during BugZapper meetings Jul 01 12:56
jlaska | did you have any news or recommendations on that front? Jul 01 12:56
Viking-Ice | debugging visualization seems complete https://fedoraproject.org/wiki/Reporting_virtualization_bugs Jul 01 12:57
Viking-Ice | mean virtualization Jul 01 12:57
jlaska | yeah, markmc and the virt team have done a great job outlining that page Jul 01 12:58
jlaska | it came in handy during the virt test day ... and hopefully will for the F12 virt testday Jul 01 12:58
Viking-Ice | oh yeah I should mentioned selinux Jul 01 12:58
Viking-Ice | https://fedoraproject.org/wiki/Design/SETroubleshootUsabilityImprovements Jul 01 12:58
Viking-Ice | Notice the report Bug <-- Jul 01 12:59
jlaska | are there things here you think the QA team should be considering? Jul 01 13:00
wwoods | yeah the "report bug" dialog there is closer to what you'd want to see from a bug-reporting client Jul 01 13:00
Viking-Ice | The only thing flawed in this design is calling the button Report Bug instead of Report Alert Jul 01 13:00
Viking-Ice | This is a security application and thus user might think all alerts are just bugs and start to ignore it Jul 01 13:01
Viking-Ice | Just thought to mention what's happening on selinux front for users and the great job they are doing... Jul 01 13:02
Viking-Ice | Anyway i'm done Jul 01 13:03
jlaska | thanks for the update ... anything you want to check-in on next week on this subject Jul 01 13:03
jlaska | or things you want the larger f-t-list to review/consider before next meeting? Jul 01 13:03
fhornain | ping ? Jul 01 13:04
jlaska | #topic Open Discussion Jul 01 13:04
--- | fedbot has changed the topic to: Open Discussion (Meeting topic: Fedora QA meeting) Jul 01 13:04
jlaska | We've run over, but I just wanted to toss a note out there for any feedback/input on the Fedora 11 test day survey results Jul 01 13:04
jlaska | https://www.redhat.com/archives/fedora-test-list/2009-July/msg00009.html Jul 01 13:05
Viking-Ice | nope low on time these day's.. we have to finish something next week before everybody start bailing for vacations.. Jul 01 13:05
Viking-Ice | here @ work.. Jul 01 13:05
jlaska | I've gathered the feedback to the wiki and starting to toss recommendations on the list for how we can improve F12 based on the feedback Jul 01 13:05
jlaska | Viking-Ice: no worries, understood. Jul 01 13:05
jlaska | That's all I had for today ... any other thoughts / concerns? Jul 01 13:05
fhornain | Can we use the room ? Jul 01 13:06
jlaska | fhornain: sure can, we are wrapping up Jul 01 13:06
fhornain | Thx Jul 01 13:06
jlaska | Thanks for your time today, let's keep things going on f-t-list and #fedora-qa Jul 01 13:06
jlaska | #endmeeting Jul 01 13:06
--- | fedbot has changed the topic to: Channel is used by various Fedora groups and committees for their regular meetings | Note that meetings often get logged | For questions about using Fedora please ask in #fedora | See http://fedoraproject.org/wiki/Meeting_channel for meeting schedule Jul 01 13:06
fedbot | Meeting ended Wed Jul 1 17:06:57 2009 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . Jul 01 13:06
fedbot | Minutes: http://www.scrye.com/~kevin/fedora/fedora-meeting/2009/fedora-meeting.2009-07-01-16.00.html Jul 01 13:06
fedbot | Log: http://www.scrye.com/~kevin/fedora/fedora-meeting/2009/fedora-meeting.2009-07-01-16.00.log.html Jul 01 13:06

Generated by irclog2html.py 2.7 by Marius Gedminas - find it at mg.pov.lt!