From Fedora Project Wiki

Fedora 10 Installation Test Plan

Revision history

Date Revision Comment
Template:Void4 June 2009 0.1 Initial version

Introduction

This document describes the tests that will be created and used to verify the installation of Fedora 10.

The goals of this plan are to:

  • Organize the test effort
  • Communicate the strategy, scope and priorities of the planned tests to all relevant stake-holders for their input and approval
  • Serve as a base for the test planning for future Fedora releases

Test Strategy

Instead of outlining all possible installation inputs and outputs, this test plan will focus on defining inputs and outputs at different stages in anaconda. This will also allow different tests to be performed independently during a single installation. For example, one may execute a kickstart delivery via HTTP, raid0 partitioning using 3 physical disks, and a minimal package installation on a para-virtualized xen guest all in single installation. Scenarios where the stages are dependent will be indicated as such in the test case.

Where possible, SNAKE will be used to automate and aid in reproducibility.

Test Priority

This test plan will use a 3 tier classification for test execution priority.

Tier1 is intended to verify that installation is possible on common hardware using common use cases. Verification includes:

  • Common boot media
  • Common Installation source
  • Installation using defaults installation options
  • Default Partitioning

Tier2 takes a step further to include more use cases. Tier2 verification consists of:

  • All boot media
  • All installation sources
  • All kickstart delivery methods
  • Some architecture specific verification

Lastly, Tier3 captures the remaining identified use cases:

  • More exhaustive partitioning schemes
  • More complex networking scenarios
  • More architecture specific verification
  • Network device
  • Storage device
  • Upgrade testing

Scope

Testing will include:

  • Various methods of booting the installation program
  • Manual and kickstart execution of the installation program
  • System setup performed by the installation program (networking, modprobe.conf, bootloader, runlevel)
  • Booting the installed system

Items outside the scope of this test plan include:

  • Functional verification of software installed on the system
  • Installation from media not generated by fedora release engineering

Test Pass/Fail Criteria

Important.png
FIXME
Need to add a link to the FUDCon10 rawhide acceptance criteria

Entrance criteria

  • Trees must be generated using release engineering tools (not hand crafted)
  • There must be no unresolved dependencies for packages included in the installation tree
  • There must be no dependency conflicts for packages included in the installation tree
  • Any changes in composition of the installation tree are explainable by way of bugzilla

Alpha criteria

  • Entrance criteria have been met
  • All tier#1 tests have been executed

Beta criteria

  • Alpha criteria have been met
  • All tier#1 tests pass
  • All tier#2 tests have been executed

GA criteria

  • Beta criteria have been met
  • All test tiers must pass
  • Any open defects have been documented as release notes

Test Deliverables

  • This test plan
  • A test summary document for each major milestone
  • A list of defects filed
  • Any test scripts used for automation or verification

Test Cases (Functional)

Important.png
FIXME
Need confirmation on the following list of features for F10 anaconda

The following list of features was obtained from Anaconda/Features. Test plans for these features will be designed/developed on each feature page.

Feature Owner Target completion
FirstAidKit (Improving Rescue Mode) JoelGranados / MartinSivak Fedora10
Improved netconfig UI with NetworkManager by default DavidCantrell Fedora10
Land new pyparted and port anaconda to it DavidCantrell Fedora10
Preserve /home on upgrade (bug #435402) ?? Fedora10
Save exceptions straight to bugzilla ChrisLumens Fedora10
Additional Encrypted Block Device support DaveLehman Fedora10 ?

Test Cases (Non-Functional)

Boot Methods
Tested designed to validate the bootable media
QA/TestCases/BootMethodsBootIso QA/TestCases/BootMethodsCdrom QA/TestCases/BootMethodsDvd
QA/TestCases/BootMethodsUsb QA/TestCases/BootMethodsPxeboot QA/TestCases/BootMethodsNetboot
QA/TestCases/BootMethodsXenParaVirt QA/TestCases/BootMethodsKVM QA/TestCases/BootMethodsRescueMode
Installation Source
The media booted and the installation source used aren't always the same. These tests verify installation using the described source.
QA/TestCases/InstallSourceHttp QA/TestCases/InstallSourceNfs QA/TestCases/InstallSourceFtpAnonymous
QA/TestCases/InstallSourceNfsIso QA/TestCases/InstallSourceCdrom QA/TestCases/InstallSourceDvd
QA/TestCases/InstallSourceHardDrive
Kickstart Delivery
Tests to validate acquiring a kickstart script through supported methods.
Initrd (ks=<path>) Block Device (ks=<dev>:<path>) HTTP (ks=http://<server>/<path>)
NFS (ks=nfs:<server>:<path>)
Package Sets
Designed to exercise the most common package dependency and conflict pathways.
QA/TestCases/PackageSetsDefaultPackageInstall QA/TestCases/PackageSetsMinimalPackageInstall QA/TestCases/PackageSetsEverything
Partitioning
The more common partitioning scenarios. These cases ensure that anaconda (and friends) prepare the disk for post-install booting as directed.
QA/TestCases/PartitioningExt3OnNativeDevice QA/TestCases/PartitioningExt2OnNativeDevice QA/TestCases/PartitioningUninitializedDisks
QA/TestCases/PartitioningRootfsOnLvmDevice QA/TestCases/PartitioningSwapOnLvmDevice QA/TestCases/PartitioningPreExistingLvm2Lvm2
QA/TestCases/PartitioningNoSwap QA/TestCases/PartitioningRaid0OnLvmDevice
QA/TestCases/PartitioningRootfsOnRaid1 QA/TestCases/PartitioningUsrOnRaid0 QA/TestCases/PartitioningUsrOnRaid5
QA/TestCases/PartitioningUsrOnRaid6 QA/TestCases/PartitioningPreExistingRaidRaid QA/TestCases/PartitioningRootfsOnDmraidDevice
Recovery
When stuff goes wrong ... we need to be able to handle it reasonably well.
QA/TestCases/UpdatesImgPrompt QA/TestCases/UpdatesImgViaTree QA/TestCases/UpdatesImgViaHttp
QA/TestCases/UpdatesImgViaUsb
QA/TestCases/TracebackSaveRemote QA/TestCases/TracebackDebugMode
Storage Devices
Are we probing and booting post-install properly in the following scenarios?
QA/TestCases/StorageDeviceSata QA/TestCases/StorageDeviceScsi QA/TestCases/StorageDeviceiScsi
QA/TestCases/StorageDeviceDmRaid
User Interface
Anaconda provides several user-interfaces for installation, the following cases are designed to ensure the desired interface operates as expected
QA/TestCases/UserInterfaceGraphical QA/TestCases/UserInterfaceVnc QA/TestCases/UserInterfaceText
QA/TestCases/UserInterfaceCmdline QA/TestCases/UserInterfaceTelnet

Test Environment/Configs

Hardware

  • i386
  • ppc
  • x86_64

Responsibilities

  • who's doing what

Schedule/Milestones

  • The Fedora 10 schedule is available at Releases/10/Schedule
  • Each major milestone will demand a full regression run (Alpha, Beta, PreviewRelease)

Risks and Contingencies

  • If a particular build is uninstallable, a updates.img may be used to work around any test blockers. For information on building and using an updates.img please refer to Anaconda/Updates.
Note.png
An updates.img can't always save the day
Please note, any issues affecting stage#1 loader (networking segfaults) or kernel boot issues may not be resolvable by using an updates.img. For these issues, your defect will need to be prioritized as a test blocker so that it can be addressed in a subsequent build.
  • If a build is uninstallable due to a bad kernel or driver, you may be able to work around the problem. Anaconda provides several scripts for taking a newer kernel, and using that during installation. For details, please refer to [scripts/upd-kernel]

Approvals

Date Approver Comment
Template:Void23 July 2008 User:jlaska I approve this message

References