Is anaconda broken proposal

Overview
The purpose of AutoQA installation test automation is to simplify testing and improve test efficiency. The AutoQA install test project should address the following problems:


 * The system should be easy to allow for tester and developer use
 * Have clear documentation for customizing and creating new tests
 * Support test execution using existing Fedora infrastructure services, but not require them
 * Test results are easy to verify

Scope
The Fedora installation test automation project covers the whole installation from retrieving installation media to first boot. Details are below:
 * 1) retrieve installation media
 * 2) Installation media sanity check
 * 3) Install with different system specifications, installation sources and partition etc.
 * 4) get the installation output and logs.

Problem Space
The Fedora installer is a complicated software application that often requires significant setup time to properly and efficiently test. Installer failures typically come from the following areas:

 Image Sanity 
 * 1) ISO file size is too large
 * 2) Invalid SHA256 checksum
 * 3) Invalid implanted ISO md5sum
 * 4) Install environment - anaconda has specific application, library and config file format requirements.
 * 5) Versions check

 Boot Methods
 * 1) Boot media improperly built (PXE, boot.iso, DVD, efidisk.img)
 * 2) Installer fails to boot as a KVM guest
 * 3) Installer fails to boot as a XEN guest

Install Source
 * 1) Unable to detect install.img media
 * 2) Unable to transition to stage#2 installer

Kickstart Delivery
 * 1) Ks.cfg could not be obtained from specified location (http, ftp, nfs, hd, initrd)
 * 2) Install fails to proceed in accordance with the directives in the ks.cfg file
 * 3) Install improperly sets up networking based on command-line and kickstart network parameters (boot with ,  contains  )

User Interface
 * 1) X driver problems while transitioning to graphical install
 * 2) Screen corruption during text-mode install
 * 3) VNC fails to start
 * 4) Serial console redirection improperly setup

Storage Devices
 * 1) Fail to detect existing storage device(s)
 * 2) Failure to clear stale data off of existing devices
 * 3) Unable to add iSCSI volumes

Partitioning
 * 1) Failure detecting existing partition scheme (lvm, mdraid, dmraid, luks)
 * 2) Failure when attempting to resize existing partitions
 * 3) Failures while attempting to re-use existing partitions
 * 4) Improperly clearing stale information from disks
 * 5) Unable to consistently resize an existing filesystem
 * 6) General failures while attempting to manually partition a system

Package Repository
 * 1) Unable to read metadata from package repositories (http, ftp, nfs, media)
 * 2) Failures while adding or modifying existing package repositories

Package Set
 * 1) Network timeout while retrieving packages
 * 2) Dependency problems while resolving package list
 * 3) File conflicts during package install
 * 4) Package order and install errors in
 * 5) Improperly formatted  data

Boot loader configuration
 * 1) Unable to properly detect other operating systems
 * 2) Failure while setting up chainloader for another OS

Upgrade system
 * 1) Failure to detect previously installed systems
 * 2) Errors while attempting to update bootloader configuration during upgrade
 * 3) Package upgrade errors in

Recovery
 * 1) Rescue mode fails to detect existing installations (lvm, raid, luks)
 * 2) Rescue mode fails to establish networking
 * 3) Problems saving traceback information (local disk, bugzilla, remote server)
 * 4) Anaconda unable to download and use updates.img (from install source, local media or URL)
 * 5) Unable to transition to debug mode

Proposed Solution
The project will be developed based on the AutoQA framework and use the virtualization environment (libvirt,kvm-qemu). The installation automation will be deployed by the Fedora kickstart installation. virt-install will be used to create the guest.virtio is used to get the anaconda logs. The details can be found at Is anaconda broken solution

Active Ingredients

 * AutoQA must be packaged and available in Fedora infrastructure
 * A valid backend test harness is packaged and available in Fedora infrastructure (Autotest currently being used by AutoQA)
 * Knowledge of TurboGears or TurboGears2
 * A new AutoQA test watcher may be required to initiate automated tests whenever CD and DVD ISO images are available for testing (post-iso-build)
 * Adding installer unit test execution and results display is dependent on the availability and environmental requirements of installer unit tests

Roadmap

 * Roadmap for development of auto install at:Is_anaconda_broken_roadmap
 * Status for test plan/documentation changes can be seen in the fedora-qa trac system
 * Status for test automation efforts is available in the autoqa TRAC system

Results

 * Meeting information and logs can be found at Is_anaconda_broken_meeting

Discussion Points

 * List unresolved or active discussion

Comments?
To leave a comment, use the Talk page for this proposal.

Owner
Fedora QA team