Is anaconda broken roadmap

This page provides a high-level roadmap for implementing the Is_anaconda_broken_proposal project. More detailed tasks can be found in autoqa TRAC roadmap. We follow these steps to define the methods by which we initiate testing

First, in order to provide a consistent and documented test approach, the existing Fedora Install test plan will be revisited. The test plan will be adjusted to ensure proper test coverage for the failure scenarios listed above. Existing test cases will be reviewed for accuracy. New test cases will be created using the Template:QA/Test_Case template. Finally, the test plan will be adjusted to match the improved Fedora Release Criteria. This includes adjusting the test case priority to match milestone criteria.

Next, in order to reduce the setup/execution time, improve efficiency and to provide test results on a more consistent basis, a subset of test cases will be chosen for automation. Tests will be written in python and will be developed and executed on a system supporting KVM virtualization. Test scripts will be responsible for preparing a virtual install environment, initiating a kickstart install and validating the results. Once an initial batch of tests exist, they will be formally integrated into the AutoQA project.

Last, a method will be developed for collecting test results into a single test result matrix. Results may be posted to the wiki directly, or a custom turbogears application may be needed to display results. The results will be easily accessible for testers and the installer development team.

The project will be divided into several phases.

Phase#1 - proof of concept

 * The following test cases are selected:
 * QA:Rawhide_Acceptance_Test_Plan
 * DVD.iso Installation
 * Boot.iso/Netinst.iso Installation
 * Live.iso Installation
 * upgrade an exiting system
 * system with basic video driver
 * Rescue installed system
 * QA:Testcase_Memtest86
 * Investigate methods for leveraging GUI automation to aid in automating applicable test cases
 * Close the Virt Viewer of the guest
 * Investigate methods for leveraging GUI automation to aid in automating applicable test cases
 * Close the Virt Viewer of the guest
 * Investigate methods for leveraging GUI automation to aid in automating applicable test cases
 * Close the Virt Viewer of the guest
 * Investigate methods for leveraging GUI automation to aid in automating applicable test cases
 * Close the Virt Viewer of the guest
 * Investigate methods for leveraging GUI automation to aid in automating applicable test cases
 * Close the Virt Viewer of the guest
 * Close the Virt Viewer of the guest
 * Close the Virt Viewer of the guest

Implement the selected test cases

 * create a function to report mediakit sanity status
 * create a function to report mediakit sanity status
 * Boot.iso/Netinst.iso Installation
 * system sanity
 * QA:Testcase_Mediakit_ISO_Size
 * QA:Testcase_Mediakit_ISO_Checksums
 * QA/TestCases/BootMethodsBootIso
 * QA/TestCases/InstallSourceBootIso
 * Live.iso Installation
 * system sanity
 * QA:Testcase_Mediakit_ISO_Size
 * QA:Testcase_Mediakit_ISO_Checksums
 * upgrade an exiting system
 * perform a default installation of the previous release
 * install the current release
 * QA:Testcase_Anaconda_User_Interface_Basic_Video_Driver
 * QA:Testcase_Anaconda_rescue_mode
 * Memory test - memtest86
 * create a function to report mediakit sanity status
 * create a function to report mediakit sanity status
 * Boot.iso/Netinst.iso Installation
 * system sanity
 * QA:Testcase_Mediakit_ISO_Size
 * QA:Testcase_Mediakit_ISO_Checksums
 * QA/TestCases/BootMethodsBootIso
 * QA/TestCases/InstallSourceBootIso
 * Live.iso Installation
 * system sanity
 * QA:Testcase_Mediakit_ISO_Size
 * QA:Testcase_Mediakit_ISO_Checksums
 * upgrade an exiting system
 * perform a default installation of the previous release
 * install the current release
 * QA:Testcase_Anaconda_User_Interface_Basic_Video_Driver
 * QA:Testcase_Anaconda_rescue_mode
 * Memory test - memtest86
 * create a function to report mediakit sanity status
 * create a function to report mediakit sanity status
 * Boot.iso/Netinst.iso Installation
 * system sanity
 * QA:Testcase_Mediakit_ISO_Size
 * QA:Testcase_Mediakit_ISO_Checksums
 * QA/TestCases/BootMethodsBootIso
 * QA/TestCases/InstallSourceBootIso
 * Live.iso Installation
 * system sanity
 * QA:Testcase_Mediakit_ISO_Size
 * QA:Testcase_Mediakit_ISO_Checksums
 * upgrade an exiting system
 * perform a default installation of the previous release
 * install the current release
 * QA:Testcase_Anaconda_User_Interface_Basic_Video_Driver
 * QA:Testcase_Anaconda_rescue_mode
 * Memory test - memtest86
 * create a function to report mediakit sanity status
 * Boot.iso/Netinst.iso Installation
 * system sanity
 * QA:Testcase_Mediakit_ISO_Size
 * QA:Testcase_Mediakit_ISO_Checksums
 * QA/TestCases/BootMethodsBootIso
 * QA/TestCases/InstallSourceBootIso
 * Live.iso Installation
 * system sanity
 * QA:Testcase_Mediakit_ISO_Size
 * QA:Testcase_Mediakit_ISO_Checksums
 * upgrade an exiting system
 * perform a default installation of the previous release
 * install the current release
 * QA:Testcase_Anaconda_User_Interface_Basic_Video_Driver
 * QA:Testcase_Anaconda_rescue_mode
 * Memory test - memtest86
 * create a function to report mediakit sanity status
 * Boot.iso/Netinst.iso Installation
 * system sanity
 * QA:Testcase_Mediakit_ISO_Size
 * QA:Testcase_Mediakit_ISO_Checksums
 * QA/TestCases/BootMethodsBootIso
 * QA/TestCases/InstallSourceBootIso
 * Live.iso Installation
 * system sanity
 * QA:Testcase_Mediakit_ISO_Size
 * QA:Testcase_Mediakit_ISO_Checksums
 * upgrade an exiting system
 * perform a default installation of the previous release
 * install the current release
 * QA:Testcase_Anaconda_User_Interface_Basic_Video_Driver
 * QA:Testcase_Anaconda_rescue_mode
 * Memory test - memtest86
 * upgrade an exiting system
 * perform a default installation of the previous release
 * install the current release
 * QA:Testcase_Anaconda_User_Interface_Basic_Video_Driver
 * QA:Testcase_Anaconda_rescue_mode
 * Memory test - memtest86

Automate remainder of test cases from the install test plan

 * Rawhide Acceptance Test Plan
 * QA:Anaconda_bootloader_setup_test_case
 * QA:X_basic_display_test_case
 * QA:X_basic_input_handling_test_case
 * QA:Network_basic_test_case
 * QA:Yum_simple_update_test_case
 * DVD installation
 * QA:Testcase_Additional_Http_Repository
 * QA:Testcase_Additional_Ftp_Repository
 * QA:Testcase_Additional_Mirrorlist_Repository
 * QA:Testcase_Additional_NFS_Repository
 * Boot.iso/netinst.iso installation
 * QA:Testcase_Http_Repository
 * Live.iso installation
 * QA:TestCases/Install_Source_Live_Image

Implement the general test cases

 * QA:Testcase_Anaconda_User_Interface_Cmdline
 * QA:Testcase_Anaconda_updates.img_via_URL, QA:Testcase_Anaconda_updates.img_via_installation_source, QA:Testcase_Anaconda_updates.img_via_local_media
 * QA:Testcase_Anaconda_save_traceback_to_remote_system, QA:Testcase_Anaconda_save_traceback_to_bugzilla, QA:Testcase_Anaconda_save_traceback_to_disk, QA:Testcase_Anaconda_traceback_debug_mode
 * upgrade
 * QA:Testcase_Anaconda_Upgrade_New_Bootloader
 * QA:Testcase_Anaconda_Upgrade_Skip_Bootloader
 * QA:Testcase_Anaconda_Upgrade_Update_Bootloader
 * QA:Testcase_Anaconda_Upgrade_Encrypted_Root
 * QA:Testcase_Anaconda_Upgrade_Skip_Bootloader_Text_Mode
 * QA:Testcase_Anaconda_Upgrade_Update_Bootloader_Text_Mode


 * preupgrade
 * QA:Testcase_Preupgrade
 * QA:Testcase_Preupgrade_from_older_release

Create kick start database to cover test cases that can be covered with kick start

 * QA:Testcase_Anaconda_User_Interface_Graphical
 * QA:Testcase_Anaconda_User_Interface_Text
 * QA:Testcase_Anaconda_User_Interface_VNC
 * different packages selections: QA/TestCases/PackageSetsDefaultPackageInstall, QA/TestCases/PackageSetsMinimalPackageInstall
 * different partition: QA:Testcase_Anaconda_autopart_install, QA:Testcase_Anaconda_autopart_(encrypted)_install, QA:Testcase_Anaconda_autopart_(shrink)_install, QA:Testcase_Anaconda_autopart_(use_free_space)_install, QA/TestCases/PartitioningExt4OnNativeDevice, QA/TestCases/PartitioningExt3OnNativeDevice, QA/TestCases/PartitioningXFSOnNativeDevice, QA/TestCases/PartitioningBtrFSOnNativeDevice, QA/TestCases/PartitioningNoSwap, QA:Testcase_Partitioning_On_Software_RAID
 * QA:Testcase_Anaconda_rescue_mode

Phase#3 - integration

 * Identify test event triggers which will be used to automatically initiate testing
 * Create appropriate control files and test wrappers to allow for scheduling tests through AutoQA (see Writing_AutoQA_Tests)
 * Develop or update AutoQA test event hooks to accommodate new test events (see Writing_AutoQA_Hooks)
 * Implement initial test result dashboard intended to eventually replace the wiki test matrix. The dashboard will also support FAS user test result submission.  This will likely rely on