This document describes the tests that will be created and used to verify the functions/components of Fedora 12.
The goals of this plan are to:
- Organize the test effort
- Communicate the planned tests to all relevant stake-holders for their input and approval
- Serve as a base for the test planning for future Fedora 12 releases
Instead of outlining all possible installation inputs and outputs, this test plan will focus on defining inputs and outputs at different stages in anaconda. This will also allow different tests to be performed independently during a single installation. For example, one may execute a kickstart delivery via HTTP, raid0 partitioning using 3 physical disks, and a minimal package installation on a para-virtualized xen guest all in single installation. Scenarios where the stages are dependent will be indicated as such in the test case.
New features of Fedora 12
As with Fedora 11, Fedora 12 will bring us some new features. The following list outlines the larger changes that affect installation. Test plans for these features will be designed/developed on each feature page.
Additional features outside the scope of testing can be found at:
- The Fedora 12 release schedule is available at Releases/12/Schedule
- Each major milestone(Beta,Preview,etc..) will demand a full regression run
This test plan will use a 3 tier classification for test execution priority.
Tier1 is intended to verify that installation is possible on common hardware using common use cases.
- Common boot media
- Common Installation source
- Installation using defaults installation options
- Default Partitioning
Tier2 takes a step further to include more use cases. Tier2 verification consists of:
- All boot media
- All installation sources
- All kickstart delivery methods
- Some architecture specific verification
Tier3 captures the remaining identified use cases:
- More exhaustive partitioning schemes
- More complex networking scenarios
- More architecture specific verification
- Network device
- Storage device
- Upgrade testing
Test Pass/Fail Criteria
The milestone release of Fedora 12 should conform these criteria:
- Trees must be generated using release engineering tools (not hand crafted)
- There must be no unresolved dependencies for packages included in the installation tree
- There must be no dependency conflicts for packages included in the installation tree
- Any changes in composition of the installation tree are explainable by way of bugzilla
- Entrance criteria have been met
- All test tiers have have been executed
- All tier#1 tests pass
- Alpha criteria have been met
- All tier#2 tests pass
- Beta criteria have been met
- All test tiers must pass
- Any open defects have been documented as release notes
Scope and Approach
Testing will include:
- Manually execute test cases using a rawhide boot or live image media
- Where possible, the use of virtualization is encouraged to allow for greater test coverage and improve failure diagnosis
- Automated execution of tests is encouraged using kickstarts designed to execute specific test cases. At this time, kickstarts are not provided for every test case.
Items outside the scope of this test plan include:
- Functional verification of software installed on the system
- Installation from media not generated by fedora release engineering
- This test plan
- A test summary document for each major milestone
- A list of defects filed
- Any test scripts used for automation or verification
Testing will execute test cases to verify installation of Fedora 12 on different hardware platforms and gather installation test feedback.
For Fedora 12,test cases will be executed on these hardware platforms:
Fedora QA team members are responsible for executing this test plan. Contributions from Rawhide testers and other interested parties are encouraged.
Risks and Contingencies
- If no physical media is available for testing, it is acceptable to use a
boot.isoor PXE boot images from a current Rawhide release. For more information, see Releases/Rawhide#Direct_Rawhide_install.
Reporting Bugs and Debugging Problems
If defects/problems are encountered, please go ahead and file the bugs following the guide below:
- jlaska 19:08, 3 August 2009 (UTC)