User:Jlaska/Fedora 15 Installation automation ideas

This page is intended to outline the current state of automated installer tests, identify possible short-term objects and identify task owners.

Existing Installer Tests
This section lists tests or scripts that are used now during manual verification of the installer.


 * 1)  - runs multiple tests
 * 2) Build test - runs
 * 3) Pylint test - Runs
 * 4)  - Storage unit testsuite in the   branch of autoqa.git
 * 5)  script - Script that creates a mock chroot and creates install images using  (see also instructions)
 * 6)  - AutoQA test script that does a basic automated kickstart install

Update AutoQA storage unit testsuite

 * Scope
 * The patchset in the clumens autoqa branch may need to be refreshed to master and re-run to check for proper operation. This idea involves rebasing the patch to master, and re-running the tests to ensure they continue to work.  In addition, we need to add more storage tests.


 * Skill Level
 * Advanced


 * Owner
 * Chris Lumens


 * Status
 * Test-result-pass.png - Completed and merged into master (see autoqa clumens branch)

Create a bugzilla query (or script) to identify storage unit testsuite candidates

 * Scope
 * Work with anaconda-devel to identify a pattern to identify bug reports that may have traceback information that can be used to craft new storage unit tests. This task does not involve creating the tests, only identifying a bugzilla query or script that can query bugzilla for test candidates.


 * Skill Level
 * Beginner


 * Owner
 * unassigned

Create AutoQA test based on script

 * Scope
 * Currently, anaconda-devel uses a script called to run  and  against anaconda source.  This script would be pulled into AutoQA as a new test.  Eventually, the test would be run after git commits, but in the short-term, could run on a nightly basis.


 * Skill Level
 * Moderate


 * Owner
 * James Laska and Chris Lumens


 * Status
 * Test-result-pass.png - completed and merged into master (see autoqa clumens branch)

Create AutoQA post-git-commit watcher

 * Scope
 * To support triggering tests based on git commits, create a new AutoQA watcher.


 * Skill Level
 * Moderate


 * Owner
 * Test-result-pass.png - completed and merged into master (see autoqa clumens branch)

Automate Installation test matrix

 * Scope
 * Automate installation test matrix (see Is_anaconda_broken_proposal). This is definitely not a short-term deliverable, and is dependent on several tasks already on this page.  Open to additional suggestions/ideas on short-term tasks that are in line with this larger goal.


 * Skill Level
 * Advanced


 * Owner
 * unassigned

Create AutoQA test based on script

 * Scope
 * Create a new AutoQA test using the script that runs pungi inside a mock chroot.  This test is intended to run after a koji build of  or it's dependent packages.


 * Skill Level
 * Moderate


 * Owner
 * jlaska


 * Status
 * Test-result-pass.png - completed and merged into master (see autoqa clumens branch)

Verify and integrate VirtIO into rats_install test

 * Scope
 * With, we now can monitor installation progress without requiring an active network to transmit logs using minimon. Installing without networking allows test automation that more accurately mimics non-network installations.  This task involves testing  to confirm that the required support exists as well as updating  to use virtio for installation progress monitoring.


 * Skill Level
 * Moderate


 * Owner
 * Hongqing


 * Status
 * Test-result-inprogress.gif - INPROGRESS

Update AutoQA test detection to handle subdirectories

 * Scope
 * Since various anaconda tests are being added now, it makes sense to organize them all as subdirectories of a tests/anaconda/ directory. To support this, AutoQA needs to be able to traverse down into directories and look for subdirectories containing tests.


 * Skill Level
 * Moderate


 * Owner
 * unassigned