From Fedora Project Wiki
(Updated status)
(Merged into master, yay!)
 
(2 intermediate revisions by the same user not shown)
Line 24: Line 24:
: [[User:Clumens|Chris Lumens]]
: [[User:Clumens|Chris Lumens]]
; Status
; Status
: [[File:Test-result-inprogress.gif|link=]] - Code complete, pending merge into ''master'' (see [http://git.fedorahosted.org/git/?p=autoqa.git;a=shortlog;h=refs/heads/clumens autoqa ''clumens'' branch])
: [[File:Test-result-pass.png|link=]] - Completed and merged into ''master'' (see [http://git.fedorahosted.org/git/?p=autoqa.git;a=shortlog;h=refs/heads/clumens autoqa ''clumens'' branch])


=== Create a bugzilla query (or script) to identify storage unit testsuite candidates ===
=== Create a bugzilla query (or script) to identify storage unit testsuite candidates ===
Line 45: Line 45:
: [[User:Jlaska|James Laska]] and [[User:Clumens|Chris Lumens]]
: [[User:Jlaska|James Laska]] and [[User:Clumens|Chris Lumens]]
; Status
; Status
: [[File:Test-result-inprogress.gif|link=]] - code complete, pending push to ''master'' (see [http://git.fedorahosted.org/git/?p=autoqa.git;a=shortlog;h=refs/heads/clumens autoqa clumens branch])
: [[File:Test-result-pass.png|link=]] - completed and merged into ''master'' (see [http://git.fedorahosted.org/git/?p=autoqa.git;a=shortlog;h=refs/heads/clumens autoqa clumens branch])


=== Create AutoQA post-git-commit watcher ===
=== Create AutoQA post-git-commit watcher ===
Line 54: Line 54:
: <span style="color:orange;font-weight:bold;">Moderate</span>
: <span style="color:orange;font-weight:bold;">Moderate</span>
; Owner
; Owner
: [[File:Test-result-inprogress.gif|link=]] - preliminary support exists in [http://git.fedorahosted.org/git/?p=autoqa.git;a=shortlog;h=refs/heads/clumens autoqa ''clumens'' branch]
: [[File:Test-result-pass.png|link=]] - completed and merged into ''master'' (see [http://git.fedorahosted.org/git/?p=autoqa.git;a=shortlog;h=refs/heads/clumens autoqa ''clumens'' branch])


=== Automate Installation test matrix ===
=== Automate Installation test matrix ===
Line 72: Line 72:
: <span style="color:orange;font-weight:bold;">Moderate</span>
: <span style="color:orange;font-weight:bold;">Moderate</span>
; Owner
; Owner
: ''unassigned''
: [[User:jlaska|jlaska]]
; Status
: [[File:Test-result-pass.png|link=]] - completed and merged into ''master'' (see [http://git.fedorahosted.org/git/?p=autoqa.git;a=shortlog;h=refs/heads/clumens autoqa clumens branch])


=== Verify {{bz|576439}} and integrate VirtIO into rats_install test ===
=== Verify {{bz|576439}} and integrate VirtIO into rats_install test ===
Line 81: Line 83:
: <span style="color:orange;font-weight:bold;">Moderate</span>
: <span style="color:orange;font-weight:bold;">Moderate</span>
; Owner
; Owner
: ''unassined''
: [[User:Hongqing|Hongqing]]
; Status
: [[File:Test-result-inprogress.gif|link=]] - ''INPROGRESS''  


=== Update AutoQA test detection to handle subdirectories ===
=== Update AutoQA test detection to handle subdirectories ===

Latest revision as of 17:32, 14 January 2011

This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

This page is intended to outline the current state of automated installer tests, identify possible short-term objects and identify task owners.

Existing Installer Tests

This section lists tests or scripts that are used now during manual verification of the installer.

  1. checkbot.sh - runs multiple tests
    1. Build test - runs ./autogen.sh && ./configure.sh && make
    2. Pylint test - Runs tests/pylint/runpylint.sh
  2. tests/anaconda_storage - Storage unit testsuite in the clumens branch of autoqa.git
  3. compose-tree script - Script that creates a mock chroot and creates install images using pungi (see also instructions)
  4. tests/rats_install/install.py - AutoQA test script that does a basic automated kickstart install

Project Ideas

Update AutoQA storage unit testsuite

Scope
The patchset in the clumens autoqa branch may need to be refreshed to master and re-run to check for proper operation. This idea involves rebasing the patch to master, and re-running the tests to ensure they continue to work. In addition, we need to add more storage tests.
Skill Level
Advanced
Owner
Chris Lumens
Status
- Completed and merged into master (see autoqa clumens branch)

Create a bugzilla query (or script) to identify storage unit testsuite candidates

Scope
Work with anaconda-devel to identify a pattern to identify bug reports that may have traceback information that can be used to craft new storage unit tests. This task does not involve creating the tests, only identifying a bugzilla query or script that can query bugzilla for test candidates.
Skill Level
Beginner
Owner
unassigned


Create AutoQA test based on checkbot.sh script

Scope
Currently, anaconda-devel uses a script called checkbot.sh to run pylint and make against anaconda source. This script would be pulled into AutoQA as a new test. Eventually, the test would be run after git commits, but in the short-term, could run on a nightly basis.
Skill Level
Moderate
Owner
James Laska and Chris Lumens
Status
- completed and merged into master (see autoqa clumens branch)

Create AutoQA post-git-commit watcher

Scope
To support triggering tests based on git commits, create a new AutoQA watcher.
Skill Level
Moderate
Owner
- completed and merged into master (see autoqa clumens branch)

Automate Installation test matrix

Scope
Automate installation test matrix (see Is_anaconda_broken_proposal). This is definitely not a short-term deliverable, and is dependent on several tasks already on this page. Open to additional suggestions/ideas on short-term tasks that are in line with this larger goal.
Skill Level
Advanced
Owner
unassigned

Create AutoQA test based on compose-tree script

Scope
Create a new AutoQA test using the compose-tree script that runs pungi inside a mock chroot. This test is intended to run after a koji build of anaconda or it's dependent packages.
Skill Level
Moderate
Owner
jlaska
Status
- completed and merged into master (see autoqa clumens branch)

Verify RHBZ #576439 and integrate VirtIO into rats_install test

Scope
With RHBZ #576439, we now can monitor installation progress without requiring an active network to transmit logs using minimon. Installing without networking allows test automation that more accurately mimics non-network installations. This task involves testing RHBZ #576439 to confirm that the required support exists as well as updating rats_install/install.py to use virtio for installation progress monitoring.
Skill Level
Moderate
Owner
Hongqing
Status
- INPROGRESS

Update AutoQA test detection to handle subdirectories

Scope
Since various anaconda tests are being added now, it makes sense to organize them all as subdirectories of a tests/anaconda/ directory. To support this, AutoQA needs to be able to traverse down into directories and look for subdirectories containing tests.
Skill Level
Moderate
Owner
unassigned