From Fedora Project Wiki

< QA

(Added testopia bug linking "Open Issue")
m (Adding testopia search to 'open issues')
Line 55: Line 55:
* Testopia Component list - Should the testopia instance just re-use [http://bugzilla.redhat.com bugzilla] product and components?
* Testopia Component list - Should the testopia instance just re-use [http://bugzilla.redhat.com bugzilla] product and components?
* Linking testrun to bugs - testopia supports linking bugs to a test run.  It appears to be looking in the current db for bugs, does it support looking into another bugzilla db for a valid bug#?
* Linking testrun to bugs - testopia supports linking bugs to a test run.  It appears to be looking in the current db for bugs, does it support looking into another bugzilla db for a valid bug#?
* Searching - It's difficult to differentiate bugzilla search.cgi vs testopia search.cgi. 
** For example, how can I search for test cases matching a specific component?
** Or, show me all test runs against the Beta build


= Lessons Learned =
= Lessons Learned =

Revision as of 14:03, 16 June 2008

Summary

The Fedora QA team is interested in evaluating Testopia as a test planning and results recording solution.

Background

During the Fedora9 campaign, the fedora wiki was used to planning, recording and presenting test execution status. Examples can be found:

While the information gathered was very helpful, and in the case of swfdec, was used to support dropping the proposed feature from F9, manually editing the wiki was not optimal. Additionally, tracking which test cases to include in a particular test run was easy to mis-type. Manually adding results to a wiki-style table entry was also easy to mis-type.

Pro's

  • Recorded test cases provided manual steps needed for execution and verification
  • The test plan outlined a plan of attack and provided a prioritization and organization for the proposed test cases
  • Wiki worked well for test plan and case content
  • Low barrier to entry ... a wiki is straight forward and uncomplex

Con's

  • Gather alternative views of test metrics from the wiki is not possible
  • Wiki not so great for hard metrics (date, tester, pass/fail execution metrics)

Tasks

STATUS OWNER(s) TASK
DEFERRED Create testopia.fedorahosted.org site (trac, tickets, git, mailing list etc...)
COMPLETE User:dmalcolm Rough packaging of testopia so that we can install and test on publictest2
INPROGRESS User:dmalcolm, User:jstanley Package review for testopia RPM

See bug#450013

COMPLETE User:jkeating Apply for fedora infrastructure sponsorship. Jesse has volunteered to be our sponsor
COMPLETE User:dmalcolm Apply for shared "publictest environment" to house proof of concept

Available at https://publictest2.fedoraproject.org/bugzilla

INPROGRESS User:jlaska Migrate existing F9 data into testopia for sample data set

See Testopia Plan#2

Open issues

  • Testopia Component list - Should the testopia instance just re-use bugzilla product and components?
  • Linking testrun to bugs - testopia supports linking bugs to a test run. It appears to be looking in the current db for bugs, does it support looking into another bugzilla db for a valid bug#?
  • Searching - It's difficult to differentiate bugzilla search.cgi vs testopia search.cgi.
    • For example, how can I search for test cases matching a specific component?
    • Or, show me all test runs against the Beta build

Lessons Learned

The following list outlines important issues uncovered during the testopia proof of concept: