Fedora 15 QA Retrospective

Introduction

This page is intended to gather feedback from the Fedora QA community on things that worked well and things that could have been better with the testing of Fedora 15. The feedback will be used as a basis for identifying areas for improvement for Fedora 15 testing. Any thoughts, big or small, are valuable. If someone already provided feedback similar to what you'd like to add, don't worry ... add your thoughts regardless.

For any questions or concerns, send mail to test@lists.fedoraproject.org.

Providing feedback

• Gwjasu - I like ____ about the new ____ process

Adding feedback is fairly straight forward. If you already have a Fedora account ...

1. Login to the wiki
2. Select [Edit] for the appropriate section below.
3. Add your feedback using the format:

   * ~~~ - I like ____ about the new ____ process

4. When done, Submit your changes

Otherwise, if you do not have a Fedora account, follow the instructions below ...

1. Select the appropriate page for your feedback...
   • Something that worked well
   • Something that didn't work well
   • Anything on your QA wishlist
2. Add your feedback using the format:

   * ~~~ - I like ____ about the new ____ process

3. When done, Submit your changes

Feedback

Could have been better

1. jlaska - Alpha-TC wasn't branched - F15 Alpha test compose was created using pre-branched content. This was unexpected, but it appears that there is no clear guidance/documentation on what is expected (see https://fedorahosted.org/rel-eng/ticket/4399)
2. jsmith - Mass rebuild not included in schedule - The mass rebuild timing wasn't ideal, it landed at the same time of the branch. While release engineering did an outstanding job resolving rebuild issues in a timely manner, the schedule didn't account for the mass rebuild, and test composes were delayed.
3. jlaska - insufficient live testing - Because of RHBZ #672265 and RHBZ #676904 we didn't get a lot of testing on the live images prior to, and during the Alpha test composes. As a result, once those issues were fixed, we found 3 additional Alpha blockers
   • RHBZ #679107 - TypeError: argument 2 to map() must support iteration
   • RHBZ #663294 - RuntimeError: XOpenDisplay failed }} - TypeError: argument 2 to map() must support iteration
   • RHBZ #672030 - AttributeError: 'NoneType' object has no attribute 'format' }} - TypeError: argument 2 to map() must support iteration
4. jlaska - missed first 2 blocker review meetings - Not sure what else to say ... we forgot about these meetings. With FUDCon and PTO, the first two completely fell off my radar.
5. jlaska - i18n - i18n/l10n continues to highlight problems early on in the release. F-15-Alpha slipped one week due to RHBZ #676827 (keyboard with german layout doesn't work in gdm). We need release criteria for LANG/keymap issues.
6. jlaska - Missing criteria - When proposing and publishing release criteria, use caution when making changes. Particularly when release criteria pages are already available for the next release. You must be sure that any changes made to 41 are copied forward to 42. This was problem for Fedora 15 Alpha with the desktop artwork criteria. The artwork criteria was added to the Fedora 14 Alpha criteria page, but the Fedora 15 Alpha criteria page was already created. The new criteria were not carried forward, and Fedora 15 Alpha released with the incorrect artwork.