|
|
| (3 intermediate revisions by one other user not shown) |
| Line 1: |
Line 1: |
| =Introduction=
| | #REDIRECT [[QA/SOP_Release_Validation_Test_Event]] |
| Installation testing is a very important service offered by the [[QA]] team. It provides coordinated and focused testing of official Fedora installation media prior to a release milestone (e.g. Alpha, Beta, Preview, GA). This page describes the steps involved when coordinating an installation test event. For any concerns, please contact the [[QA]] group.
| |
| | |
| =Set a date for install test =
| |
| The install test event is held one or two weeks before the release date for any given milestone. See [http://poelstra.fedorapeople.org/schedules/f-{{FedoraVersion||next}}/f-{{FedoraVersion||next}}-quality-tasks.html quality task schedule] for specific dates for {{FedoraVersion|long|next}}. DVD/CD/LiveCD images for download need to be made available before this date, and this time should not conflict with other important Fedora events.
| |
| | |
| =Create test result page=
| |
| The test result page is used to gather test result of installation against Fedora Release Candidate builds. This page mainly includes test cases which need to be executed and tables to gather the results. Refer to [[QA:Create_Install_Test_Result_Page]] for guidance to create such a page using [[QA:{{FedoraVersion|long|next}}_Install_Results_Template]].
| |
| | |
| = Announce install test event =
| |
| Currently, we announce install test events to the [https://admin.fedoraproject.org/mailman/listinfo/test-announce test-announce]. The announcement mail should provide enough information related to the test focus areas. For example:
| |
| | |
| * Introduction of this test event (date, what to test, known bugs, etc...)
| |
| * What test cases need to be executed
| |
| * Download address of DVD/CD/LiveCD images
| |
| * How and where to add test results
| |
| * Contact information of QA members who are available on test event and can help testers who encounter problems.
| |
| | |
| It is best to announce this event on the mailing list at least 3 days in advance. This should give testers sufficient time to arrange their calendars and prepare a test environment. It is also a good idea to send a reminder e-mail the day before the test.
| |
| | |
| Try to take timezones into account, to maximize convenience for testers from different regions or countries.
| |
| | |
| =Provide help during install test=
| |
| During install test event, lots of people will be involved, including experienced users and new comers. Make sure the QA folks whose contact information was announced to mailing list in this test event are available during the testing period. They will provide assistance to those who encounter issues.
| |
| QA people should be available at：
| |
| * IRC: #fedora-qa on irc.freenode.net
| |
| * Mailing list: [https://admin.fedoraproject.org/mailman/listinfo/test test list]
| |
| * Reference of ways to communicate at [[Communicate]]
| |
| | |
| =Report and Summary=
| |
| | |
| After testing has completed， send a test summary to the [https://admin.fedoraproject.org/mailman/listinfo/test-announce test-announce] mailing list. The test summary is intended to keep testers informed as to what was accomplished by their testing, whether there are any remaining tasks and to recognize key contributors.
| |
| | |
| Report at should include the following information:
| |
| ; Test status
| |
| : A summary about what has been tested, what's remaining to accomplish, what issues were faced during test and whether we have achieved testing aim
| |
| ; Bugs reported
| |
| : List filed bugs and reported issues which were found during test. Bug list including bug id, status, summary, etc...
| |
| : The following command can be used to generate a list of bugs:
| |
| <pre>curl "https://fedoraproject.org/wiki/Test_Results:Current_Installation_Test" | \
| |
| grep -o "bugzilla\.redhat\.com.*[=\/][0-9]\{6\}" | \
| |
| grep -o "[0-9]\{6\}" | tr '\n' ',' | sed 's|,$||' | \
| |
| xargs bugzilla query --outputformat="%{bug_id} %{bug_status} %{resolution} - %{short_desc}" -b</pre>
| |
| | |
| {{admon/note|Got <code>python-bugzilla</code>?|This command requires the {{package|python-bugzilla}} package be installed. You may need to weed out any 'false' references to pre-existing bugs which are mentioned on the page for some reason.}}
| |
| | |
| ; Analyze results and assess risk
| |
| : Analyze the results and provide an assessment where additional risk areas may exist
| |
| ; Thanks to all testers
| |
| : Testers have volunteered their time, it never hurts to be thankful for their contributions.
| |
| : To generate a list of testers who contributed to the wiki, you can use the following command:
| |
| <pre>curl "https://fedoraproject.org/w/index.php?action=raw&title=Test_Results:Fedora_14_Final_RC1_Install" 2>/dev/null | \
| |
| grep -A999999 "^= Test Matrix =$" | grep "{{result" | sed -e "s|{{\(result[^}]*\)}}|\r\1 |g" | tr '|' ' ' | \
| |
| gawk '$3{print $3} !$3{print $2}' | sort | uniq -c | sort -bgr</pre>
| |
| | |
| [[Category:QA SOPs]]
| |
| [[Category:Install Test]]
| |
