From Fedora Project Wiki

Line 20: Line 20:


== Determine when the test should run ==
== Determine when the test should run ==
# Roger examines the list of [http://git.fedorahosted.org/git/?p=autoqa.git;a=tree;f=hooks available hooks] to see which events are monitored. For each hook he reads the ''README'' file to read a short description of the hook.
# Roger examines the ''autoqa.cron'' file to see how often there is a check for new events for different hooks by default.
# If Roger needs more detailed information he examines the watchers (''watch-*.py'' files in hooks' directories) to see the current implementation. By seeing the source code he can find out some important details.
# Roger now knows which hook (which event type) is most suitable to trigger his new test.


== Integrating the test into AutoQA ==
== Integrating the test into AutoQA ==

Revision as of 13:06, 16 December 2009

Introduction

This page is intended to be a comprehensive list of all the ways people will interact with AutoQA. All items will be focused on outlining the steps required to accomplish a specific task. This page will detail activities for test developers, administrators, release engineers and package maintainers. This page follows a similar design to the Fedora_Talk_Admin_Cases and related pages.

  • If the use case works today, there should be a link to a wiki page explaining how to do it
  • If the use case does not exist yet, there should be a link to an AutoQA Ticket.

Test Developer Use Cases

The following use cases are aimed at Roger. Roger is active in the QA community and would like to help QA team to catch as many problems as possible before they reach Fedora users. He decided to write a new test and integrate it into AutoQA framework to increase the test coverage.

Decide what to test

  1. Roger examines the list of available tests to see which tests are already executed. For each test he looks into the control file to read a short description of that test.
  2. Roger examines the list of available hooks to see which events are monitored. For each hook he reads the README file to read a short description of the hook and the testlist file to see which tests are executed when corresponding event happens.
  3. Roger finds a problem area that is not currently covered by tests and he would like to cover it. Roger takes into consideration if the new test would be triggered by an existing hook (easier) or a new hook would have to be written (harder). He also considers if the new test would utilize existing external tools and scripts (like rpmlint, repoclosure, etc.) or new tools would have to be written to achieve the goal.
  4. Roger now has a clear idea what to test and by which means to achieve it.

Automate a test

Determine when the test should run

  1. Roger examines the list of available hooks to see which events are monitored. For each hook he reads the README file to read a short description of the hook.
  2. Roger examines the autoqa.cron file to see how often there is a check for new events for different hooks by default.
  3. If Roger needs more detailed information he examines the watchers (watch-*.py files in hooks' directories) to see the current implementation. By seeing the source code he can find out some important details.
  4. Roger now knows which hook (which event type) is most suitable to trigger his new test.

Integrating the test into AutoQA

  1. Installing the required software -- [1]
  2. Building a control file for his test according to Writing_AutoQA_Tests#The_control_file

Verifying the test works

Contributing the test

  1. Roger has a new test, which seems to be working well. He wants to publish it in the AutoQA project upstream.
  2. Roger creates a new ticket in AutoQA Trac and provides a patch or a link to his git branch there.
  3. Roger can also start a discussion in autoqa-devel mailing list about his work.
  4. Roger waits until his new work is accepted.


Administrator Use Cases

These use cases are aimed at Nancy. Nancy is a member of the Fedora Infrastructure team and has been asked to help the AutoQA project with sysadmin tasks that require access to infrastructure systems and tools.

Create a AutoQA system from scratch

  1. Her first task is to prepare a new Fedora system
  2. Once the system is prepared, Nancy reads and follows the instructions for how to Install_and_configure_autotest
  3. Now, Nancy must install AutoQA packages - [2]

Add a new test system to AutoQA

  1. Nancy first started by setting up an AutoQA system from scratch (see #Create a AutoQA system from scratch)
  2. Next, Nancy wants to add one or more systems that Autotest will use as test systems. She does so by following the instructions at How_to_add_autotest_clients

Recover a failed test system

Remove a test system

Update puppet configuration

Package Maintainer Use Cases

Ned is the maintainer of several packages in Fedora. After having dealt with a several reoccurring bugs in the last round of updates to his packages, Ned would like to write some tests to help capture the failures before they happen.

View existing test coverage

Write a test

  1. See #Write_a_test perhaps?
  2. Where do they store the tests? In CVSDist?

Run the test(s) manually

Test for proper integration of the test(s)

Subscribe for notifications to a selected test (or test/package combination)

Received notification of test failure ... need more details?

  1. FIXME - What is required for a client setup?
  2. FIXME