QA/Meetings/20111128

From FedoraProject

Jump to: navigation, search

Contents

Attendees

  • adamw (117)
  • kparal (37)
  • j_dulaney (28)
  • tflink (23)
  • wwoods (11)
  • jskladan (7)
  • pschindl (5)
  • zodbot (4)
  • brunowolff (2)
  • nirik (1)

Agenda

Previous meeting follow-up

  • on the question of whether we need code hooks in the anaconda UI rewrite to aid automated testing: kparal mailed anaconda-devel-list and wwoods / jlaska directly, but response so far seems thin
    • various ideas discussed for moving the topic forward, kparal will continue to try
  • j_dulaney is still happy with the test day co-ordination role, adamw is available for help
  • adamw has nearly finished the retrospective wrap-up, several tickets filed so far

Revised F17 schedule confirmation

F16 retrospective activity

  • this was shelved until next week as adamw has not finished preparing it and the meeting was running long

AutoQA update

  • team is looking into some issues found in testing of the new autoqa version
  • most infrastructure has now been upgraded from F14
  • resultsdb to begin testing in staging after autoqa 0.7 is deployed

Open floor

Action items

  • kparal to continue to look at whether we need hooks in anaconda UI code for testing, will see if anyone in RH's desktop QA team has experience
  • adamw to send the (revised schedule) proposal along to rbergeron

IRC Log

adamw #startmeeting Fedora QA Meeting 16:00
zodbot Meeting started Mon Nov 28 16:00:35 2011 UTC. The chair is adamw. Information about MeetBot at http://wiki.debian.org/MeetBot. 16:00
zodbot Useful Commands: #action #agreed #halp #info #idea #link #topic. 16:00
adamw #meetingname fedora-qa 16:00
zodbot The meeting name has been set to 'fedora-qa' 16:00
adamw #chair tflink j_dulaney 16:00
zodbot Current chairs: adamw j_dulaney tflink 16:00
adamw #topic roll call 16:00
* nirik is lurking around if he can help with anything. 16:01
* j_dulaney is ready to rock-and-roll! 16:01
adamw good morning everybody, welcome to another stonking week of fedora qa 16:01
* pschindl is here 16:01
* jskladan enters with evil laughter 16:01
j_dulaney Uh oh... 16:02
adamw hey hey hey, that's my job 16:02
* tflink is here 16:03
adamw #topic muahahahahahahaahahahah! 16:03
adamw ahem 16:03
* brunowolff is here 16:03
adamw #topic previous meeting follow-up 16:03
adamw hellooo, everybody. okay, let's see what we had hanging around from last week 16:03
adamw "kparal to try and co-ordinate all involved in writing installer autoqa tests together to determine if we need any hooks in the new anaconda code" 16:04
adamw well, it looks like kparal isn't around 16:04
adamw does anyone know his status on that? 16:04
j_dulaney #info I think adamw fired kparal last week... 16:04
jskladan I know that kparal sent out some email to the anaconda guys 16:04
adamw heh 16:04
jskladan but I did not see any reply, apart of the ... 16:05
adamw jskladan: thanks 16:05
* jskladan is looking for it 16:05
adamw aha, here he is 16:05
* kparal joins in last 16:05
kparal at last 16:05
adamw kparal: just in time for your starring role! 16:05
adamw we're on your action item from last week: "kparal to try and co-ordinate all involved in writing installer autoqa tests together to determine if we need any hooks in the new anaconda code" 16:05
kparal oh yes 16:05
adamw where'd you get with that? 16:06
kparal I sent an email to anaconda-devel 16:06
* kparal looking for link 16:06
pschindl jskladan is out for a moment (gnome-shell is fell) 16:06
adamw what? gnome shell crashed?! never! 16:06
pschindl *fallen :) 16:06
j_dulaney Not Gnome Shell! 16:07
kparal #link https://www.redhat.com/archives/anaconda-devel-list/2011-November/msg00112.html 16:07
pschindl yep, today for second time (propably some poblem with weather applet) 16:07
kparal pschindl: there are no applets, don't you know? 16:07
kparal anyway, I started the discussion and a few people responded 16:07
pschindl sry. extension 16:07
kparal but it doesn't seem like a huge discussion 16:08
kparal I hoped for some knowledgeable people to respond, like wwoods or jlaska, because they had experience with the previous test 16:08
adamw okay 16:08
* j_dulaney only sees clumens 16:08
adamw if you're worried the right people aren't replying, probably best poke them directly 16:08
kparal anyway at least we know the current rats-install test doesn't work :) 16:08
adamw the thread may have just skipped under their radar 16:08
kparal I CCed them directly I believe 16:09
adamw ah okay 16:09
adamw #info kparal mailed anaconda-devel-list and wwoods / jlaska directly, but response so far seems thin 16:09
j_dulaney Should we get Máirín Duffy in on this? 16:09
kparal they know we can respond if they have some questions regarding autoqa 16:10
adamw not sure mo would have much input? 16:10
j_dulaney Since she's doing a lot of the design work 16:10
kparal and that's probably the most important 16:10
adamw j_dulaney: yeah, but that's at a very high level 16:10
j_dulaney True 16:10
tflink I think she's on anaconda-devel but I'm not sure that testable hooks into anaconda is so much of a design level 16:10
kparal nope 16:10
adamw j_dulaney: essentially mo draws workflow diagrams, it's still the anaconda team that writes the actual code :) (no insult intended either side) 16:10
* j_dulaney understands now 16:11
adamw kparal: well, having a line of communication open here is great, but this doesn't sound like something anaconda team will be actively asking us about 16:11
adamw it's more a 'we'll put in anything you need but we need you to tell us what it is' deal 16:11
tflink yeah 16:11
adamw i'd feel more comfortable if we did at least have a decent idea whether we need stuff in there or not 16:11
kparal I don't know and finding out is quite an extensive task 16:12
tflink I'm still not clear on what it is that we would want to do 16:12
adamw hrm. 16:12
kparal I hoped just to start the discussion 16:12
adamw yeah 16:12
tflink but I'm not sure my questions about purpose are as appropriate for here 16:13
adamw i guess all we can really do if we don't have any clear ideas is re-ping it 16:13
* jskladan aaaaand I'm back 16:13
adamw let's try putting it this way: is the uncertainty here most likely to be a symptom of us *not* needing any special code in anaconda for (auto)qa purposes? 16:14
tflink no, I don't think so 16:14
tflink at least not for me 16:14
adamw well, shucks. :) 16:14
j_dulaney I'm thinking it would make our life easier 16:15
tflink my question is "what are we going to try to automate?" 16:15
* j_dulaney is just not sure *what* code is needed, not that it *is* needed 16:15
kparal ideally anaconda guys will write their own test suite and hook it up in autoqa. so they should know best 16:15
j_dulaney kparal: Do they know that they need to do this? 16:15
adamw did you ask them specifically if they're planning to do that? 16:15
tflink it sounds like they aren't really sure what would be helpful 16:15
kparal they already have some test suite in autoqa 16:16
tflink at least from clumens' response 16:16
j_dulaney kparal: Won't the new Anaconda render that obsolete? 16:18
kparal j_dulaney: clumens responded something like they will need to update it 16:18
tflink j_dulaney: I don't think so. AFAIK, there is no GUI interface in those tests 16:18
kparal once they start to receive emails with test results they will be more active about it I hope 16:18
adamw so i'm still kinda looking for a clear way forward here: is the best we can do wait and hope? or re-ping specific people and hope they reply this time? 16:19
kparal oh, and I don't think we are talking about GUI testing, right? we are talking about anaconda testing 16:19
j_dulaney How much besides GUI/Workflow is changing? 16:19
tflink kparal: I thought we were talking about GUI testing since that's the major change for F17, I think 16:19
j_dulaney How much will that change the backend stuff? 16:19
adamw i think a lot of it is looking like re-organization rather than change exactly 16:20
adamw they're moving to a hub / spoke design rather than a serial one 16:21
adamw (aiui) 16:21
kparal there might be some misunderstandings here. I believe the new GUI is best tested by humans, and autoqa can be used to test the plumbing 16:22
kparal if you think we should automate GUI testing, well, I don't think we have the manpower to do that ATM 16:22
adamw no, that's not it 16:22
tflink I think we might need to add some clarification to the discussion, then 16:22
adamw yeah. 16:23
tflink I'm reading clumens' responses as talking about automated GUI testing 16:23
adamw gimme a sec to back up and check the history 16:23
adamw oh, kay 16:23
adamw sorry, my mistake: we are on UI stuff here 16:23
kparal as I wrote to clumens, the thing they can do is to allow to control the whole workflow programatically, without clicking buttons. then it can be tested 16:23
adamw but the thing is that even though we can't automate UI testing *now*, if we need fundamental code in there to make it possible in future, we should ask now 16:24
adamw because they aren't going to retrofit it. 16:24
tflink kparal: that's still testing the GUI even if it isn't through screenshots and clicking 16:24
adamw so yeah, we are specifically on UI here, and i guess part of the problem is that we don't have any UI tests so it's hard to know what we'd need 16:24
kparal that means we have to have someone who understands UI testing 16:25
tflink specifically, automated and scripted UI testing 16:25
adamw yeah. i think what we're missing here is the experience of trying to write automated UI tests against an existing codebase and going 'damn, wouldn't it be great if i could XXX' 16:25
j_dulaney Indeed 16:26
adamw kparal: tflink: do you know of anyone who fits the bill? 16:26
kparal I can talk to the RH desktop team and ask 16:27
adamw yeah, that might be a good way to go. 16:27
kparal we have a few guys sitting here in Brno 16:27
adamw sorry to drag this item out so far, but it'd be nice to have a definite plan. 16:27
tflink adamw: no not really, I've heard about building in testable interfaces but have never dealt with them myself and I don't know of anyone who has 16:27
j_dulaney Would anyone outside of Fedora have this experience? 16:28
* wwoods reading scrollback 16:29
adamw j_dulaney: well, probably? but we'd need someone who'd talk to us and be able to explain what we'd need 16:29
adamw oh hey, wwoods 16:29
adamw wwoods: executive summary: clumens has told us that if we want hooks for automated testing in the new UI code we should ask for them now so they get baked in at the start, but we're really not sure what would be useful as we don't really have any automated UI testing experience 16:29
adamw wwoods: kparal was hoping you might have a few thoughts 16:30
wwoods generally UI automation seems to use a11y 16:31
adamw (accessibility, for those playing along at home) 16:32
wwoods right. I can't remember what the current libraries are for this stuff 16:32
wwoods but from the anaconda side - we just have to make sure we write a proper accessibility-enabled UI 16:32
wwoods with the UI rewrite we're taking every step to use the standard widgets where possible 16:33
adamw okay 16:33
wwoods and we're trying to make sure the UI can be run on a normal system in a nice test mode 16:33
wwoods so it should be pretty easy to mess around and see how/if you can automate it 16:33
wwoods I think we can commit to making sure the UI is a11y-enabled (whatever that might entail) 16:33
adamw okay 16:34
adamw i think we're not necessarily going to get much farther in here 16:34
wwoods but I'll definitely bring it up with the team and we'll talk further on the mailing list(s) and/or put something in the UI rewrite docs 16:34
* j_dulaney thinks that it comes down to what tests we want to run 16:34
adamw kparal: okay if i action this back to you to try and follow up this week? 16:34
kparal yes 16:35
adamw okay 16:35
wwoods basically what it comes down to is: nearly automated GUI testing uses the a11y stuff, so if you want to be able to automate the UI itself, we need to have a11y support 16:35
kparal wwoods: just please respond with your thoughts to my email in anaconda-devel, thanks 16:36
adamw #action kparal to continue to look at whether we need hooks in anaconda UI code for testing, will see if anyone in RH's desktop QA team has experience 16:36
wwoods kparal: will do. 16:36
adamw thanks wwoods 16:36
adamw okay, let's try and blow through the rest of this fast :) 16:36
adamw "j_dulaney to take a shot at co-ordinating F17 test days, thanks! " 16:36
adamw j_dulaney: I sent you a sort of 'getting started' guide, did you get that? 16:37
j_dulaney I did 16:37
adamw cool 16:37
* j_dulaney has been on vacation 16:37
j_dulaney I'll get to it this afternoon 16:37
adamw that's fine 16:37
adamw just wanted to make sure communications are functioning properly etc :) 16:37
adamw alright, so nothing further necessary there 16:38
adamw #info j_dulaney is still happy with the test day co-ordination role, adamw is available for help 16:38
adamw "adamw to get retrospective tickets filed and summary mailed this week" 16:38
adamw i'm not quite done with this, but close 16:38
adamw i'm not gonna go into huge detail because it'd just duplicate the summary mail, but i've been filing tickets on our trac and releng's, and sending out a few ml discussions 16:39
adamw most of the tickets are just up for grabs, so do feel free to grab any you feel like taking a shot at 16:39
adamw QA tickets so far are 256 through 260 16:40
adamw #info adamw has nearly finished the retrospective wrap-up, several tickets filed so far 16:40
adamw anyone have any concerns / questions there or should we roll along? 16:41
j_dulaney Rolling on the river 16:42
adamw rolling rolling rolling 16:42
adamw rawhiiiiiiiiide! 16:42
adamw "adamw to start an ml thread proposing changes to the tc dates for f17 schedule to be sent to rbergeron " 16:42
adamw we've got an agenda item for that later, so let's table it 16:42
adamw aaand here it comes! 16:42
adamw #topic Revised F17 schedule confirmation 16:43
j_dulaney Ponies! 16:43
adamw so i did indeed start a thread proposing revisions to the TC drop dates for f17 16:43
adamw https://lists.fedoraproject.org/pipermail/test/2011-November/104583.html 16:43
adamw it looks like everyone was happy with that, so i'm planning to ask the berge to put it in place - just wanted to check if there were any last-minute issues anyone saw 16:43
j_dulaney Not here 16:44
adamw speak now or forever hold thy peace 16:44
adamw okay! 16:45
adamw #agreed proposed revisions to f17 schedule look good 16:45
adamw #action adamw to send the proposal along to rbergeron 16:45
adamw #topic F16 retrospective activity 16:45
adamw oh, yeah, i did want to take a few minutes just to flag up the QA retrospective tickets i filed so far so everyone knows the kinds of stuff we need to take care of ahead of f17 alpha cycle starting up 16:46
adamw it might be best to shelve this for next week though as there's some stuff that might start up interesting discussions but we're kinda long already 16:47
adamw is that okay? or was anyone really hoping for a nice long meeting? :) 16:47
tflink works for me 16:48
* j_dulaney is good 16:48
adamw #agreed this topic shelved till next week since not all tickets are yet filed and this meeting is running long 16:48
adamw #topic AutoQA update 16:48
adamw anything exciting from the world of autoqa, folks? 16:48
tflink still looking to get the new version of autoqa deployed 16:49
kparal we are just in the process of reviewing staging results from the past week and the weekend 16:49
kparal we found some weird issues 16:49
kparal it seems mainly autotest related, but are not sure 16:50
kparal *we 16:50
kparal I updated documentation about autoqa configuration 16:50
adamw #info team is looking into some issues found in testing of the new autoqa version 16:50
kparal if we don't find anything critical I would like to deploy _soon_ 16:51
kparal who's got more updates? 16:51
jskladan pschindl & I are once again working on the ResultsDB (querying api and the frontend) 16:51
tflink most of the production infrastructure has been updated to not run on F14 any more 16:52
jskladan and IIRC, after we deploy the new version, we'll start testing ResultsDB on our Staging setup 16:52
adamw #info most infrastructure has now been upgraded from F14 16:52
adamw #info resultsdb to begin testing in staging after autoqa 0.7 is deployed 16:52
adamw sounds good 16:53
adamw okay, that's all from autoqa? 16:54
tflink I think so 16:54
adamw cool 16:54
adamw thanks 16:54
adamw #topic open floor 16:54
adamw aaand we made it - any other stuff? 16:54
* adamw notes that you can always add agenda topics to the wiki page ahead of time, which will likely give them more time than the 'open floor' slot 16:55
adamw PSA ends! 16:55
j_dulaney No one was fired? 16:55
j_dulaney PSA? 16:55
tflink public service announcement 16:56
adamw j_dulaney: i can fire you if you think it would help. 16:56
j_dulaney I've already been fired 16:56
adamw hey, that doesn't stop me. ask tflink, he must've been fired dozens of times now. 16:58
* tflink has lost count 16:58
brunowolff FYI rawhide is a bit rough now. gnome-shell has a soname issue, so it will be hard to test fresh builds. Hopefully this gets fixed soon. 16:58
adamw usually does. 16:59
adamw okay, sounds like we're about done here 16:59
adamw thanks for coming, everyone 16:59
adamw #endmeeting 16:59

Generated by irclog2html.py 2.8 by Marius Gedminas - find it at mg.pov.lt!