User:Jlaska/Draft3

From FedoraProject

< User:Jlaska(Difference between revisions)
Jump to: navigation, search
m (Corrected table syntax)
m (Add questions section)
 
(38 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Key ==
+
{{header|qa}}
 +
{{draft}}
  
See the table below for a sample format for test results.
+
This page will outline criteria for success for a Fedora AMI release
  
{| class="wikitable sortable" border="0"
+
= Release Criteria =
! Test Result !! Explanation
+
|-
+
| {{result|none}}
+
| ''Untested'' - This test has not been run, and is available for anyone to contribute feedback.
+
|-
+
| {{result|pass|liam}}
+
| ''Passed'' - The test has been run and the tester determine the test met the ''expected results''
+
|-
+
| {{result|inprogress|adamwill}}
+
| ''Inprogress'' - An inprogress result is often used for tests that take a long time to execute.  Inprogress results should be temporary and change to pass, fail or warn.
+
|-
+
| {{result|fail|jlaska|XYZ|ZXY}}
+
| ''Failed'' - Indicates a failed test.  A link to a bug must be provided.  See [[Template:Result]] for details on providing bug information. <references/>
+
|-
+
| {{result|warn|rhe}} <ref>Brief description about the warning status</ref>
+
| ''Warning'' - This test completed and met the ''expected results'' of the test, but other issues were encountered during testing that warrant attention.<references/>
+
|-
+
| {{result|pass|rhe}} {{result|warn|kparal}}
+
| ''Multiple results'' - More people can easily provide results to a single test case.
+
|-
+
| style="background:lightgrey;"| {{testresult/none}} {{result|none}}
+
| ''Unsupported'' - An unsupported test or configuration.  No testing is required.
+
|-
+
|}
+
  
== Priority ==
+
In addition to the existing [[Fedora Release Criteria]], the following criteria apply to EC2 Fedora images.  A <span style="color:green; font-weight:bold;">good</span> AMI '''must'''...
 +
# allow non-root user login
 +
# have SELinux enabled and ''enforcing'' by default
 +
# not contain any package dependency problems
 +
# not include any packages built outside of Fedora infrastructure
  
The explanation of test case priority is available at [[QA:Fedora_13_Install_Test_Plan#Test_Priority]].
+
= How to Test =
  
== Test Matrix ==
+
To ''create'' an EC2 image, one needs...
 +
# an amazon account
 +
# the {{package|python-boto}} libraries installed
 +
# a python script to tell Amazon to build an instance
 +
# the ability to SSH into a machine to run tests, etc.
  
{| class="wikitable sortable" border="1"
+
= Open Questions =
! Release Level !! Variation !! Test Area !! Test Case !! i386 !! x86_64 !! References
+
# Who is responsible for creating and building EC2 images?
|-
+
#:  
| Alpha
+
 
| boot.iso
+
= Additional Reading =
| Image Sanity
+
* [[Publishing_image_to_EC2]]
| [[QA:Testcase_Mediakit_ISO_Size]]
+
* [[Cloud_SIG]]
| {{result|none}}
+
* [[Cloud_SIG/EC2_Images]]
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| boot.iso
+
| Image Sanity
+
| [[QA:Testcase_Mediakit_ISO_Checksums]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| boot.iso
+
| Boot Methods
+
| [[QA/TestCases/BootMethodsBootIso]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| boot.iso
+
| Image Source
+
| [[QA/TestCases/ImageSourceBootIso]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| boot.iso
+
| Install Source
+
| [[QA/TestCases/InstallSourceHttp]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| boot.iso
+
| Package Set
+
| [[QA/TestCases/PackageSetsDefaultPackageInstall]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| DVD iso
+
| Image Sanity
+
| [[QA:Testcase_Mediakit_ISO_Size]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| DVD iso
+
| Image Sanity
+
| [[QA:Testcase_Mediakit_ISO_Checksums]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| DVD iso
+
| Image Sanity
+
| [[QA:Testcase_Mediakit_Repoclosure]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| DVD iso
+
| Image Sanity
+
| [[QA:Testcase_Mediakit_FileConflicts]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| DVD iso
+
| Boot Methods
+
| [[QA/TestCases/BootMethodsDvd]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| DVD iso
+
| Image Source
+
| [[QA/TestCases/ImageSourceDvd]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| DVD iso
+
| Install Source
+
| [[QA/TestCases/InstallSourceDvd]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| DVD iso
+
| Package Set
+
| [[QA/TestCases/PackageSetsDefaultPackageInstall]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| CD iso
+
| Image Sanity
+
| [[QA:Testcase_Mediakit_ISO_Size]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| CD iso
+
| Image Sanity
+
| [[QA:Testcase_Mediakit_ISO_Checksums]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| CD iso
+
| Image Sanity
+
| [[QA:Testcase_Mediakit_Repoclosure]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| CD iso
+
| Image Sanity
+
| [[QA:Testcase_Mediakit_FileConflicts]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| CD iso
+
| Boot Methods
+
| [[QA/TestCases/BootMethodsCdrom]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| CD iso
+
| Image Source
+
| [[QA/TestCases/ImageSourceCdrom]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| CD iso
+
| Install Source
+
| [[QA/TestCases/InstallSourceCdrom]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| CD iso
+
| Package Set
+
| [[QA/TestCases/PackageSetsDefaultPackageInstall]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| Live ISO
+
| Image Sanity
+
| [[QA:Testcase_Mediakit_ISO_Size]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| Live ISO
+
| Image Sanity
+
| [[QA:Testcase_Mediakit_ISO_Checksums]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| Live ISO
+
| Image Sanity
+
| [[QA:Testcase_Mediakit_Repoclosure]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| Live ISO
+
| Image Sanity
+
| [[QA:Testcase_Mediakit_FileConflicts]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| Live ISO
+
| Install Source
+
| [[QA:TestCases/Install Source Live Image]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| PXE images
+
| Boot Methods
+
| [[QA/TestCases/BootMethodsPxeboot]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Partitioning
+
| [[QA/TestCases/PartitioningUninitializedDisks]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Partitioning
+
| [[QA:Testcase_Anaconda_autopart_install]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Partitioning
+
| [[QA:Testcase_Anaconda_autopart_(encrypted)_install]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Partitioning
+
| [[QA:Testcase_Anaconda_autopart_(shrink)_install]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Partitioning
+
| [[QA:Testcase_Anaconda_autopart_(use_free_space)_install]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Partitioning
+
| [[QA/TestCases/PartitioningExt4OnNativeDevice]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Final
+
| General
+
| Partitioning
+
| [[QA/TestCases/PartitioningExt3OnNativeDevice]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Final
+
| General
+
| Partitioning
+
| [[QA/TestCases/PartitioningExt2OnNativeDevice]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Final
+
| General
+
| Partitioning
+
| [[QA/TestCases/PartitioningNoSwap]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Final
+
| General
+
| Partitioning
+
| [[QA:Testcase_Partitioning_On_Software_RAID]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| User Interface
+
| [[QA:Testcase_Anaconda_User_Interface_Graphical]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| User Interface
+
| [[QA:Testcase_Anaconda_User_Interface_Text]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| User Interface
+
| [[QA:Testcase_Anaconda_User_Interface_VNC]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| User Interface
+
| [[QA:Testcase_Anaconda_User_Interface_Cmdline]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| User Interface
+
| [[QA:Testcase_Anaconda_User_Interface_serial_console]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Final
+
| General
+
| User Interface
+
| [[QA:Testcase_Anaconda_User_Interface_Telnet]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| User Interface
+
| [[QA:Testcase_Anaconda_rescue_mode]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Install Source
+
| [[QA/TestCases/InstallSourceFtpNonAnonymous]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Install Source
+
| [[QA/TestCases/InstallSourceNfs]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Final
+
| General
+
| Install Source
+
| [[QA/TestCases/InstallSourceNfsIso]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Final
+
| General
+
| Install Source
+
| [[QA/TestCases/InstallSourceHardDrive]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Repository
+
| [[QA:Testcase Additional Ftp Repository]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Repository
+
| [[QA:Testcase Additional Http Repository]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Repository
+
| [[QA:Testcase Additional Mirrorlist Repository]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Repository
+
| [[QA:Testcase Additional NFS Repository]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Memory Test
+
| [[memtest86+]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Boot Methods
+
| [[QA/TestCases/BootMethodsXenParaVirt]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Boot Methods
+
| [[QA/TestCases/BootMethodsKVM]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Boot Methods
+
| [[QA:Testcase_Boot_Methods_efidisk.img]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Kickstart Delivery
+
| [[QA/TestCases/KickstartKsHttpServerKsCfg]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Kickstart Delivery
+
| [[QA/TestCases/KickstartKsFilePathKsCfg]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Kickstart Delivery
+
| [[QA/TestCases/KickstartKsHdDevicePathKsCfg]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Kickstart Delivery
+
| [[QA/TestCases/KickstartKsNfsServerPathKsCfg]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Recovery
+
| [[QA:Testcase_Anaconda_updates.img_via_URL]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Recovery
+
| [[QA:Testcase_Anaconda_updates.img_via_installation_source]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Recovery
+
| [[QA:Testcase_Anaconda_updates.img_via_local_media]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Recovery
+
| [[QA:Testcase Anaconda save traceback to remote system]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Recovery
+
| [[QA:Testcase Anaconda save traceback to bugzilla]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Recovery
+
| [[QA:Testcase Anaconda save traceback to disk]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Final
+
| General
+
| Recovery
+
| [[QA:Testcase Anaconda traceback debug mode]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Storage Devices
+
| [[QA:Testcase_install_to_SATA_device]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Storage Devices
+
| [[QA:Testcase_install_to_SCSI_device]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Alpha
+
| General
+
| Storage Devices
+
| [[QA:Testcase_install_to_PATA_device]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Storage Devices
+
| [[QA:Testcase_Install_to_BIOS_RAID]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Storage Devices
+
| [[QA:Testcase_Install_to_Hardware_RAID]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Final
+
| General
+
| Storage Devices
+
| [[QA:Testcase_Anaconda_ISCSI_No_Authentication]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Upgrade system
+
| [[QA:Testcase_Anaconda_Upgrade_New_Bootloader]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Upgrade system
+
| [[QA:Testcase_Anaconda_Upgrade_Skip_Bootloader]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Upgrade system
+
| [[QA:Testcase_Anaconda_Upgrade_Update_Bootloader]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Upgrade system
+
| [[QA:Testcase_Anaconda_Upgrade_Encrypted_Root]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Upgrade system
+
| [[QA:Testcase_Preupgrade]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
| Beta
+
| General
+
| Upgrade system
+
| [[QA:Testcase_Preupgrade_low_/boot_disk_space_to_install]]
+
| {{result|none}}
+
| {{result|none}}
+
| <references/>
+
|-
+
|}
+

Latest revision as of 17:19, 9 March 2011

QA.png


Warning (medium size).png
This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

This page will outline criteria for success for a Fedora AMI release

Contents

[edit] Release Criteria

In addition to the existing Fedora Release Criteria, the following criteria apply to EC2 Fedora images. A good AMI must...

  1. allow non-root user login
  2. have SELinux enabled and enforcing by default
  3. not contain any package dependency problems
  4. not include any packages built outside of Fedora infrastructure

[edit] How to Test

To create an EC2 image, one needs...

  1. an amazon account
  2. the Package-x-generic-16.pngpython-boto libraries installed
  3. a python script to tell Amazon to build an instance
  4. the ability to SSH into a machine to run tests, etc.

[edit] Open Questions

  1. Who is responsible for creating and building EC2 images?

[edit] Additional Reading