From Fedora Project Wiki

Revision as of 07:14, 16 June 2009 by Liam (talk | contribs) (Test Environment/Configs)

Note.png
DRAFT
This page is currently a draft. This test plan has not been finalized yet. Stay tuned ...

Introduction

This document describes the tests that will be created and used to verify the functions/components of Fedora 12.

The goals of this plan are to:

  • Organize the test effort
  • Communicate the planned tests to all relevant stake-holders for their input and approval
  • Serve as a base for the test planning for future Fedora 12 releases

Test Strategy

Instead of outlining all possible installation inputs and outputs, this test plan will focus on defining inputs and outputs at different stages in anaconda. This will also allow different tests to be performed independently during a single installation. For example, one may execute a kickstart delivery via HTTP, raid0 partitioning using 3 physical disks, and a minimal package installation on a para-virtualized xen guest all in single installation. Scenarios where the stages are dependent will be indicated as such in the test case.

Where possible, SNAKE will be used to automate and aid in reproducibility.

New features of Fedora 12

Important.png
FIXME
Need confirmation on the following list of features for Fedora 12

As Fedora 11,Fedora 12 will bring us some new features.The following list of features was obtained from Anaconda/Features. Test plans for these features will be designed/developed on each feature page.

Schedule/Milestones

  • The Fedora 12 release schedule is available at Releases/12/Schedule
  • Each major milestone(Alpha,Beta,Preview,etc..) will demand a full regression run

Test Pass/Fail Criteria

This test plan will use a 3 tier classification for test execution priority.3 tier Priority of test cases ,please refer to 3 tier priority

The milestone release of Fedora 12 should conform these criteria:

Entrance criteria

  • Trees must be generated using release engineering tools (not hand crafted)
  • There must be no unresolved dependencies for packages included in the installation tree
  • There must be no dependency conflicts for packages included in the installation tree
  • Any changes in composition of the installation tree are explainable by way of bugzilla

Beta criteria

  • Entrance criteria have been met
  • All tier1 tests have been executed

Preview criteria

  • Beta criteria have been met
  • All tier1 tests pass
  • All tier2 tests have been executed

GA criteria

  • Preview criteria have been met
  • All test tiers must pass
  • Any open defects have been documented as release notes

Scope and Approach

Testing will include:

  • Manually execute test cases using a live image of rawhide or updated rawhide system
  • Manually/kickstart boot the installation program to execute the installation
  • Automatically execute installation testing via cobbler/koan etc..,cobbler guide and scripts,please refer to cobbler

Items outside the scope of this test plan include:

  • Functional verification of software installed on the system
  • Installation from media not generated by fedora release engineering

Community Testing

Testing will verify installation against Fedora 12 milestone Release Candidate builds on different hardware platforms .

Reporting Bugs and Debugging Problems

If you catch some bugs during testing,we encourage you to go ahead and file the bugs.If you a new person here,please follow the guide below

Test Environment/Configs

For Fedora 12,test cases will be executed on these hardware platforms:

  • i386
  • ppc
  • x86_64
  • s390

Communicate

If you want to participate the testing of Fedora 12,you can get help or give feedback by any of these ways:

References

This page is intended to record QA:Fedora_12_Install_Test_Plan test results for the <<FIXME>> milestone.

Instructions

What To Test

This page tracks test results against Fedora 12 Release Candidate builds. Test results against other Fedora 12 milestones can be found at Category:Fedora_12_Test_Results.

  1. Most testing can occur using rawhide -- Consult for information on downloading and installing Rawhide
  2. Tests requiring physical media (e.g. DVD or CD images) -- [<<FIXME>> download the required ISO images].

Post Test Results

Want to share information about something you tested? Please refer to QA:Create_Install_Test_Result_Page.

Add or Remove a Test Case

  1. Please request review for your changes by publishing your test case for review to fedora-test-list.
  2. Once reviewed, make your changes to this document
  3. Finally, update QA:Fedora_12_Install_Results_Template with the same changes.

Key

Please use the following format when posting results to this page

Priority Test Case i386 ppc x86_64 References
Tier1 A sample test case [1] FASName [2] FASName [3]
  1. Indicates untested
  2. Indicates a test that has passed
  3. Indicates a failed test ... see Notes/Bug(s) for details
Tier2 Another sample test case FASName [1] [2]
  1. Indicates an inprogress or test that needs further review
  2. Indicates unsupported configuration

Priority

The explanation of test case priority is available at QA:Fedora_12_Install_Test_Plan#Test_Priority.

Test Matrix

Priority Test Area Test Case i386 ppc x86_64 References
Tier1 Image Sanity QA:Testcase_Mediakit_ISO_Size
Tier1 Image Sanity QA:Testcase_Mediakit_ISO_Checksums
Tier3 Image Sanity QA:Testcase_Mediakit_Repoclosure
Tier3 Image Sanity QA:Testcase_Mediakit_FileConflicts
Tier2 Boot Methods QA/TestCases/BootMethodsBootIso
Tier2 Boot Methods QA/TestCases/BootMethodsCdrom
Tier2 Boot Methods QA/TestCases/BootMethodsDvd
Tier2 Boot Methods QA:Testcases_Installer_Boot_Methods_Netboot_(ppc)
Tier2 Boot Methods QA/TestCases/BootMethodsPxeboot
Tier2 Boot Methods QA/TestCases/BootMethodsXenParaVirt
Tier3 Boot Methods QA/TestCases/BootMethodsKVM
Tier2 Boot Methods QA:Testcase_efidisk.img
Tier1 Install Source QA/TestCases/InstallSourceHttp
Tier1 Install Source QA/TestCases/InstallSourceNfs
Tier2 Install Source QA/TestCases/InstallSourceNfsIso
Tier2 Install Source QA/TestCases/InstallSourceCdrom
Tier2 Install Source QA/TestCases/InstallSourceDvd
Tier2 Install Source QA/TestCases/InstallSourceFtpAnonymous
Tier3 Install Source QA/TestCases/InstallSourceFtpNonAnonymous
Tier2 Install Source QA/TestCases/InstallSourceHardDrive
Tier1 Install Source QA:TestCases/Install Source Live Image
Tier2 Kickstart Delivery QA/TestCases/KickstartKsFilePathKsCfg
Tier2 Kickstart Delivery QA/TestCases/KickstartKsHdDevicePathKsCfg
Tier2 Kickstart Delivery QA/TestCases/KickstartKsHttpServerKsCfg
Tier2 Kickstart Delivery QA/TestCases/KickstartKsNfsServerPathKsCfg
Tier1 Package Sets QA/TestCases/PackageSetsDefaultPackageInstall
Tier1 Package Sets QA/TestCases/PackageSetsMinimalPackageInstall
Tier1 Partitioning QA:Testcase_Anaconda_autopart_install
Tier1 Partitioning QA:Testcase_Anaconda_autopart_(encrypted)_install
Tier2 Partitioning QA:Testcase_Anaconda_autopart_(shrink)_install
Tier2 Partitioning QA:Testcase_Anaconda_autopart_(use_free_space)_install
Tier1 Partitioning QA/TestCases/PartitioningExt4OnNativeDevice
Tier1 Partitioning QA/TestCases/PartitioningExt3OnNativeDevice
Tier2 Partitioning QA/TestCases/PartitioningExt2OnNativeDevice
Tier2 Partitioning QA/TestCases/PartitioningRootfsOnLvmDevice
Tier2 Partitioning QA/TestCases/PartitioningRootfsOnRaid1
Tier2 Partitioning QA/TestCases/PartitioningNoSwap
Tier2 Partitioning QA/TestCases/PartitioningRaid0OnLvmDevice
Tier2 Partitioning QA/TestCases/PartitioningSwapOnLvmDevice
Tier2 Partitioning QA/TestCases/PartitioningUninitializedDisks
Tier2 Partitioning QA/TestCases/PartitioningUsrOnRaid0
Tier2 Partitioning QA/TestCases/PartitioningUsrOnRaid5
Tier2 Partitioning QA/TestCases/PartitioningUsrOnRaid6
Tier3 Partitioning QA/TestCases/PartitioningPreExistingLvm2Lvm2
Tier3 Partitioning QA/TestCases/PartitioningPreExistingRaidRaid
Tier2 Recovery QA:Testcase_Anaconda_rescue_mode
Tier3 Recovery QA:Testcase_Anaconda_updates.img_via_URL
Tier3 Recovery QA:Testcase_Anaconda_updates.img_via_installation_source
Tier3 Recovery QA:Testcase_Anaconda_updates.img_via_local_media
Tier3 Recovery QA:Testcase Anaconda save traceback to remote system
Tier3 Recovery QA:Testcase Anaconda save traceback to bugzilla
Tier3 Recovery QA:Testcase Anaconda save traceback to disk
Tier3 Recovery QA:Testcase Anaconda traceback debug mode
Tier2 Storage Devices QA/TestCases/StorageDeviceSata
Tier2 Storage Devices QA/TestCases/StorageDeviceScsi
Tier2 Storage Devices QA:Testcase_install_to_PATA_device
Tier2 Storage Devices QA:Testcase Anaconda iSCSI no authentication
Tier2 Storage Devices QA:Testcase_Anaconda_partitioning_dmraid_rootfs
Tier1 User Interface QA/TestCases/UserInterfaceGraphical
Tier1 User Interface QA/TestCases/UserInterfaceText
Tier1 User Interface QA/TestCases/UserInterfaceVnc
Tier2 User Interface QA/TestCases/UserInterfaceCmdline
Tier2 User Interface QA/TestCases/UserInterfaceTelnet
Tier2 Upgrade QA:Testcase_Anaconda_Upgrade_New_Bootloader
Tier2 Upgrade QA:Testcase_Anaconda_Upgrade_Skip_Bootloader
Tier2 Upgrade QA:Testcase_Anaconda_Upgrade_Update_Bootloader
Tier2 Upgrade QA:Testcase_Anaconda_Upgrade_Encrypted_Root
Tier2 Upgrade QA:Testcase_Preupgrade
Tier2 Upgrade QA:Testcase_Preupgrade_from_older_release