Objective
This document aims to establish what testing is necessary to ensure the milestone releases of the Fedora Server product meet the desired levels of quality.
Scope
The Server/Product_Requirements_Document - henceforth, "PRD" - defines the high-level aims of the Server product, and the Server/Technical_Specification - henceforth, "TS" - provides technical detail on its design. Significant points include:
- The product will be delivered as "A network installation media (either a traditional netinst.iso or a boot.fedoraproject.org style)" and "A local installation media providing the default package set as well as any featured roles that are meaningfully installed without a network connection." (TS)
- The product will use Anaconda as its deployment mechanism (implicit in the TS)
- The product's default storage configuration will be XFS-on-LVM, which differs from at least the Workstation product (TS)
- The product will be built on top of the Base WG's output (part of overall Fedora.next plan)
- A concept of "featured server roles" is defined: "A Featured Server role is an installable component of Fedora Server that provides a well-integrated service on top of the Fedora Server platform." It is mentioned that these "featured" roles will be tested: "Initially, we will test and support only a single role per server..." A set of "Mandatory Requirements" is defined for these "featured" roles; it is reasonable to deduce that we must test the "featured" roles meet these "Mandatory" requirements. (PRD)
- Scripted and interactive installations are both considered supported, and in-place upgrades. (PRD)
- Server role deployment is supported both during installation and post-install. (PRD)
- No graphical environment is considered to be a part of the Server product. (TS)
- Various requirements for the base system are documented in the TS, several of which are likely to be held in common with the other Products and current Fedora, but some of which are specific to Server.
Proposed test coverage
It is reasonable to assume that most of our existing Fedora_Release_Criteria, with the exception of those specifically relating to desktop functionality, will be held to apply to the Server product. It is therefore equally reasonable to assume that our existing QA:Installation_validation_testing and QA:Base_validation_testing processes will apply to the Server product, and the corresponding test plans ('test matrices') will need to be run for each milestone release of the Server product.
We will also require additional Server-specific testing to fully validate the product for release. Testing the "server roles" will form the majority of this work. The mechanisms for deploying and configuring server roles both during installation and after installation will need to be tested. All of the "featured" roles for any given release will also need to be tested to ensure they meet the "mandatory requirements" defined in the PRD.
Beyond this, additional "Base validation" type tests are likely to be required to validate all the commitments of the TS which are not currently explicitly tested, e.g. the requirements for remote logging and remote updating, the requirements relating to firewall functionality, and so forth. Some of these tests may be shared with the other products and deliverables.