From Fedora Project Wiki

(update intro)
(Decide what to test)
Line 8: Line 8:
 
= Test Developer Use Cases =
 
= Test Developer Use Cases =
  
The following use cases are aimed at Roger
+
The following use cases are aimed at Roger. Roger is active in the QA community and would like to help QA team to catch as many problems as possible before they reach Fedora users. He decided to write a new test and integrate it into AutoQA framework to increase the test coverage.
  
 
== Decide what to test ==
 
== Decide what to test ==
 +
 +
# Roger examines the list of [http://git.fedorahosted.org/git/?p=autoqa.git;a=tree;f=tests available tests] to see which tests are already executed. For each test he looks into the ''control'' file to read a short description of that test.
 +
# Roger examines the list of [http://git.fedorahosted.org/git/?p=autoqa.git;a=tree;f=hooks available hooks] to see which events are monitored. For each hook he reads the ''README'' file to read a short description of the hook and the ''testlist'' file to see which tests are executed when corresponding event happens.
 +
# Roger finds a problem area that is not currently covered by tests and he would like to cover it. Roger takes into consideration if the new test would be triggered by an existing hook (easier) or a new hook would have to be written (harder). He also considers if the new test would utilize existing external tools and scripts (like rpmlint, repoclosure, etc.) or new tools would have to be written to achieve the goal.
 +
# Roger now has a clear idea what to test and by which means to achieve it.
  
 
== Automate a test ==
 
== Automate a test ==

Revision as of 10:14, 16 December 2009

Introduction

This page is intended to be a comprehensive list of all the ways people will interact with AutoQA. All items will be focused on outlining the steps required to accomplish a specific task. This page will detail activities for test developers, administrators, release engineers and package maintainers. This page follows a similar design to the Fedora_Talk_Admin_Cases and related pages.

  • If the use case works today, there should be a link to a wiki page explaining how to do it
  • If the use case does not exist yet, there should be a link to an AutoQA Ticket.

Test Developer Use Cases

The following use cases are aimed at Roger. Roger is active in the QA community and would like to help QA team to catch as many problems as possible before they reach Fedora users. He decided to write a new test and integrate it into AutoQA framework to increase the test coverage.

Decide what to test

  1. Roger examines the list of available tests to see which tests are already executed. For each test he looks into the control file to read a short description of that test.
  2. Roger examines the list of available hooks to see which events are monitored. For each hook he reads the README file to read a short description of the hook and the testlist file to see which tests are executed when corresponding event happens.
  3. Roger finds a problem area that is not currently covered by tests and he would like to cover it. Roger takes into consideration if the new test would be triggered by an existing hook (easier) or a new hook would have to be written (harder). He also considers if the new test would utilize existing external tools and scripts (like rpmlint, repoclosure, etc.) or new tools would have to be written to achieve the goal.
  4. Roger now has a clear idea what to test and by which means to achieve it.

Automate a test

Determine when the test should run

Integrating the test into AutoQA

  1. Installing the required software -- [1]
  2. Building a control file for his test according to Writing_AutoQA_Tests#The_control_file

Verifying the test works

Contributing the test

Administrator Use Cases

These use cases are aimed at Nancy. Nancy is a member of the Fedora Infrastructure team and has been asked to help the AutoQA project with sysadmin tasks that require access to infrastructure systems and tools.

Create a AutoQA system from scratch

  1. Her first task is to prepare a new Fedora system
  2. Once the system is prepared, Nancy reads and follows the instructions for how to Install_and_configure_autotest
  3. Now, Nancy must install AutoQA packages - [2]

Add a new test system to AutoQA

  1. Nancy first started by setting up an AutoQA system from scratch (see #Create a AutoQA system from scratch)
  2. Next, Nancy wants to add one or more systems that Autotest will use as test systems. She does so by following the instructions at How_to_add_autotest_clients

Recover a failed test system

Remove a test system

Update puppet configuration

Package Maintainer Use Cases

Ned is the maintainer of several packages in Fedora. After having dealt with a several reoccurring bugs in the last round of updates to his packages, Ned would like to write some tests to help capture the failures before they happen.

View existing test coverage

Write a test

  1. See #Write_a_test perhaps?
  2. Where do they store the tests? In CVSDist?

Run the test(s) manually

Test for proper integration of the test(s)

Subscribe for notifications to a selected test (or test/package combination)

Received notification of test failure ... need more details?

  1. FIXME - What is required for a client setup?
  2. FIXME