From Fedora Project Wiki
Introduction
This page is intended to gather feedback from the Fedora QA community on things that worked well and things that could have been better with the testing of Fedora 13. The feedback will be used as a basis for identifying areas for improvement for Fedora 14 testing. Any thoughts, big or small, are valuable. If someone already provided feedback similar to what you'd like to add, don't worry ... add your thoughts regardless.
For any questions or concerns, send mail to test@lists.fedoraproject.org.
Providing feedback
Adding feedback is fairly straight forward. If you already have a Fedora account ...
- Login to the wiki
- Select [Edit] for the appropriate section below.
- Add your feedback using the format:
* ~~~ - I like ____ about the new ____ process
- When done, Submit your changes
Otherwise, if you do not have a Fedora account, follow the instructions below ...
- Select the appropriate page for your feedback...
- Add your feedback using the format:
* ~~~ - I like ____ about the new ____ process
- When done, Submit your changes
Feedback
Things that went well
- 192.168.1.63 - I like the higher default 1024x768 screen resolution provided for generic display upon installation.
- jlaska - Release Criteria - Having release criteria that have been reviewed and accepted by key stake holders really expedites testing by improving bug escalation time and uncertainty regarding user impact. Instead of spending precious time debating the merits of each bug report, we can collectively debate the criteria ... and adjust as needed.
- rhe - Trac Ticket for creating milestones - efficiently track the status of isos.
- rhe - key of test results - more people can easily provide test results in one test case.
- rhe - F-13-Beta test runs - highlighted test cases that needed focus in the test plan using bold text
- rhe - F-13-Beta test runs - Use of #REDIRECT pages (e.g. Test_Results:Current_Installation_Test and Test_Results:Current_Desktop_Test) and Category links instead of real ones (e.g. Beta TC results and Beta RC results).
Could have been better
- Parijath - DNS not resolving - In my default installation, firefox can not resolve urls. When I give the ip address, it is working well. Otherwise it doesn't work. 'ping -c 3 google.com' is working again. I thought there must be some problem with SELinux or Firewall. As I am a new comer, I have disabled iptables and ip6tables and also SELinux security. Then rebooted. Even then I couldn't connect to any url from firefox. I have tried to enable all the services which relate to dns and networking. What should I do ? I opened a terminal, 'su', then typed 'firefox' ( as root) then an error message has come. It is given below ... Failed to contact configuration server; some possible causes are that you need to enable TCP/IP networking for ORBit, or you have stale NFS locks due to a system crash. See http://projects.gnome.org/gconf/ for information. (Details - 1: Failed to get connection to session: Did not receive a reply. Possible causes include: the remote application did not send a reply, the message bus security policy blocked the reply, the reply timeout expired, or the network connection was broken.)
- Prarijath - Package Update problems - Next problem is ... the system has shown about 12 updates. When I selected update, it is eternal.
- jlaska 17:36, 27 April 2010 (UTC) - Greetings Parijath. Sorry to hear of your troubles. Your best bet is to join the Fedora QA test mailing list and start communicating each issue you are experiencing. Try to keep your emails specific to one topic per mail, and just as you have done here, include any diagnostics you have performed to solve the problem.
- jlaska - freeze date between TC and RC - Because the alpha development freeze takes places after the test compose, the first alpha release candidate had a lot of change and was dead on arrival (see thread discussing changes).
- jlaska - live images - Not having daily Live images available for test left several Live installer bugs lurking until late.
- rhe - use of real event urls - Redirect link urls can be used in emails, irc topic etc instead of real event urls.
- jlaska - firstboot modules - We don't have a way to know what firstboot modules are expected, we need to document this as release criteria, or write a test to report the issue (for details, see RHBZ #574596).
- jlaska - test days - Printing test day did not have as many participants as we anticipated and hoped. How come?
- rhe - schedule - some candidates were not available as expected to be available on time. Trac tickets were created by releng every time before the events, but it is not easy to tell the status of candidates from the tickets. Is there anything else to do to make sure it's uploaded as scheduled next time?
- Kparal - test days - ABRT test day did not have large attendance. Maybe we shouldn't organize test days around Easter holidays?
- jlaska - Beta RC3 testing - Beta RC3 didn't include the correct version of
plymouth - pfrields - Schedule and QA - One-week slip for Alpha was supposed to echo down through the schedule, as documented on the Fedora 13 Alpha Release Criteria page. At least one QA team member indicated the shorter time frame contributed to inability to find and resolve Beta blockers.
- Kparal - network disabled by default - As per bug 572489 the network interfaces will be disabled by default in F13 in most cases (installations from CD/DVD/USB). That's a very unfortunate default setting, especially for Fedora newcomers. We should update our Fedora Release Criteria to ensure we can mark this problem as release blocker (probably Alpha) next time. Automatically working internet connection is so basic assumption of most users that we shouldn't ship Fedora doing the opposite.
- rhe - test plan - Though some test cases are marked as Beta priority, they still didn't block beta release.
- rhe - Virt test day - Too many test cases and features for one event, perhaps would be better as a test week or a test day with smaller focus?
- liam - Test Day - Some test cases need to be improved, none of these cases were executed in Test Day. https://fedoraproject.org/wiki/Test_Day:2010-04-08_Virtualization_VirtioSerial, I have filed a ticket to tract it(https://fedorahosted.org/fedora-qa/ticket/61 ), but did not find people to do this.Test cases seem not be written by Amit.
Wishlist
- Kparal - care about low-bandwidth testers - we should improve QA processes' accessibility even for low-bandwidth users. That means primarily use existing tools for lowering download requirements of individual release milestones: deltaiso, zsync (2).
- Kparal - test days calendar - we could create and maintain a web calendar containing all Test Days (maybe even other QA activities?), that people could add to their calendar program and be notified when a new event is happening (not everybody follows the announcements in MLs and also they are announced few days in advance so people may easily forget about it -- my case). Just another way to achieve a little higher participation.
- Pfrields - Reward testers -- we should reward repeat (or frequent) community QA/testers with a 4 or 8 GB USB key and maybe another gift. I have a small pot of money we can use for this, if Fedora QA team folks are not able to just do things like this. (I would advocate that Jlaska have a small pot of money for this, amount TBD.)
- Mcepl - boot.fedoraproject.org -- it could be helpful for everybody present to deploy test-day images to http://boot.fedoraproject.org; not sure about bandwidth requirements for that, but it shouldn't be worse than everybody downloading whole images.
- rhe - community involvement - Would love to have someone outside the core team help announce or host a planned test run (Desktop, Installation or other).
- rhe - care about low-bandwidth testers - Delta-ISO's made officially available.
- Pfrields - test day incentive - One idea, if you have "repeat testers" we might want to reward them with a 4-8 GB USB key and maybe another gift?
- Kparal - easily available links to current QA activities - I would like to see floating frame in some of our important wiki pages (like QA) containing list of all current activities. That means that the list could contain link to current installation test, current (or very near) test day, today's meeting, etc. Just all the stuff where people can get involved. The list of activities would have to be probably manually updated, but I think it should be worth it. I got this idea when I was in LiveCD environment, I opened QA wiki page, and it was nearly impossible to find a link to current installation testing. When it was impossible for me, what about the public?