AutoQA

From FedoraProject

(Difference between revisions)
Jump to: navigation, search
(move hook info to its own page, add category, add draft heading)
Line 2: Line 2:
 
AutoQA is an automated test system for Fedora. Its basic design is simple: when certain events occur (a package is built, updates are pushed, a new Rawhide build is available, etc.) autoqa launches some automated tests.
 
AutoQA is an automated test system for Fedora. Its basic design is simple: when certain events occur (a package is built, updates are pushed, a new Rawhide build is available, etc.) autoqa launches some automated tests.
  
== Events and Hooks ==
+
== Hooks ==
  
For AutoQA to respond to an event, someone needs to write a "hook" for that event. The first step in creating a new hook is, obviously, to define the event it responds to.
+
AutoQA defines "hooks" for events that we are interested in. A hook basically consists of:
 +
* A definition of the event we're watching for
 +
* A Watcher program to monitor for the event and gather arguments needed for the tests
 +
* Code to handle the arguments passed from the watcher
 +
* A list of tests to run when the hook is triggered
 +
* Template files for writing new tests for this hook
  
For example, the <code>post-repo-update</code> hook is triggered whenever a Fedora repo is updated. Obviously, any test that will run in response to this event will need to know the URL of the repo that was just updated. (This is a <i>required</i> argument.) Certain repos can't be used properly without other "parent" repos (for example, the Fedora 11 <code>updates</code> repo isn't useful without also knowing the address of the main Fedora 11 repo). So a list of "parent repos" is an <i>optional argument</i> to the tests.
+
=== Defining Events ===
  
Detailed information about implementing new hooks can be found in [[Writing AutoQA Hooks]].
+
For AutoQA to respond to an event, someone needs to create a "hook" for that event. The first step in creating a new hook is, obviously, to define the event itself.
 +
 
 +
For example, the <code>post-repo-update</code> hook is triggered whenever a Fedora repo is updated. The repo update is the event, and Obviously, any <code>post-repo-update</code> test needs to know the URL of the repo that was just updated. (This is an example of a ''required'' argument.) Certain repos can't be used properly without other "parent" repos (for example, the Fedora 11 <code>updates</code> repo isn't useful without also knowing the address of the main Fedora 11 repo). So a list of "parent repos" is an ''optional'' argument to the tests.
 +
 
 +
=== Watchers ===
 +
 
 +
The next part of a hook is the Watcher. This is a program that watches for the event and launches the autoqa harness when the event occurs. The Watcher is responsible for filling in all the optional arguments for the hook and passing that information along to autoqa.
 +
 
 +
=== Test arguments ===
 +
 
 +
The autoqa test launcher needs to know how to process the arguments gathered by the watcher
 +
 
 +
More detailed information about implementing new hooks can be found in [[Writing AutoQA Hooks]].
  
 
== The autoqa Harness ==
 
== The autoqa Harness ==
  
The autoqa harness is launched by the watchers in response to an event.
+
The autoqa harness is launched by the watchers in response to an event. It's basically just a python script, <code>/usr/bin/autoqa</code>, which handles the arguments passed by the watcher and schedules the appropriate tests to be run in autotest.
 +
 
 +
== Autotest ==
 +
 
 +
Autotest is "a framework for automated testing". It handles scheduling test jobs from the job queue, putting the test code onto the appropriate test machine, running the test, and gathering the test results for later examination. For more information see [[Autotest]].
 +
 
 +
== Tests ==
 +
 
 +
AutoQA tests consist of some test metadata, some setup and test reporting code, and the test itself - which may be a pre-existing test or a new test written specifically for AutoQA.
 +
 
 +
The test results are reported back to the [[Autotest]] system and can be examined through its web interface. Tests can also opt to report their results in other ways - commonly by sending email to the [http://fedorahosted.org/mailman/listinfo/autoqa-results autoqa-results] mailing list.
 +
 
 +
For more detailed information about AutoQA tests, see [[Writing AutoQA Tests]].
  
 
[[Category:AutoQA]]
 
[[Category:AutoQA]]

Revision as of 21:06, 25 August 2009

Warning (medium size).png
This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

AutoQA is an automated test system for Fedora. Its basic design is simple: when certain events occur (a package is built, updates are pushed, a new Rawhide build is available, etc.) autoqa launches some automated tests.

Contents

Hooks

AutoQA defines "hooks" for events that we are interested in. A hook basically consists of:

  • A definition of the event we're watching for
  • A Watcher program to monitor for the event and gather arguments needed for the tests
  • Code to handle the arguments passed from the watcher
  • A list of tests to run when the hook is triggered
  • Template files for writing new tests for this hook

Defining Events

For AutoQA to respond to an event, someone needs to create a "hook" for that event. The first step in creating a new hook is, obviously, to define the event itself.

For example, the post-repo-update hook is triggered whenever a Fedora repo is updated. The repo update is the event, and Obviously, any post-repo-update test needs to know the URL of the repo that was just updated. (This is an example of a required argument.) Certain repos can't be used properly without other "parent" repos (for example, the Fedora 11 updates repo isn't useful without also knowing the address of the main Fedora 11 repo). So a list of "parent repos" is an optional argument to the tests.

Watchers

The next part of a hook is the Watcher. This is a program that watches for the event and launches the autoqa harness when the event occurs. The Watcher is responsible for filling in all the optional arguments for the hook and passing that information along to autoqa.

Test arguments

The autoqa test launcher needs to know how to process the arguments gathered by the watcher

More detailed information about implementing new hooks can be found in Writing AutoQA Hooks.

The autoqa Harness

The autoqa harness is launched by the watchers in response to an event. It's basically just a python script, /usr/bin/autoqa, which handles the arguments passed by the watcher and schedules the appropriate tests to be run in autotest.

Autotest

Autotest is "a framework for automated testing". It handles scheduling test jobs from the job queue, putting the test code onto the appropriate test machine, running the test, and gathering the test results for later examination. For more information see Autotest.

Tests

AutoQA tests consist of some test metadata, some setup and test reporting code, and the test itself - which may be a pre-existing test or a new test written specifically for AutoQA.

The test results are reported back to the Autotest system and can be examined through its web interface. Tests can also opt to report their results in other ways - commonly by sending email to the autoqa-results mailing list.

For more detailed information about AutoQA tests, see Writing AutoQA Tests.