Revision as of 23:26, 3 November 2016

The QA team develops tools to make testing easier and more repeatable. We have several projects ranging from some that are pretty simple and new developer friendly to complex and challenging stuff at the cutting edge of the field. Contact Tim Flink or the qa-devel mailing list if you're interesting in helping to build new tools for Fedora QA.

QA tools

Test Automation

Taskotron: Taskotron is our current framework for automatically executing a number of Fedora related tests. Both Fedora infrastructure generic and Fedora packages specific tests should be supported. Package maintainers should have an easy way of providing tests for their packages. This framework obsoleted AutoQA. You can see Taskotron in action.

ResultsDB: A generic storage of (automated) test results. Designed to be lightweight, simple, scalable and fast. It's used by Taskotron.

ExecDB: This database with a front-end allows us to track tasks which are scheduled, running and finished in Taskotron. It provides a simple way to navigate people (mostly admin and developer-type folks) into different corners of the Taskotron infrastructure - BuildBot job details, ResultsDB results, logs and artifacts directories. Read some design concepts.

openQA: We use this tool (developed by our friends at SUSE, now with contributions from us too) to automate certain release test cases, especially installation with Anaconda. We develop Fedora tests for openQA, along with a scheduler and some other tools. Also see the upstream home page.

Beaker: Beaker is an up and coming fully automated test framework that provides bare metal hardware provisioning including console and power management, and job scheduling and execution. Learn more at Beaker homepage.

Release validation testing

python-wikitcms, relval and testdays: A Python library and CLI based upon it for performing various release validation testing tasks in the Wikitcms 'test management system' (the set of wiki pages in which release validation test results are stored). testdays can perform a few basic operations on Test Day pages. relvalconsumer is a Fedmsg consumer which creates validation test events.

check-compose: A script for running informational checks on a Fedora compose and printing or mailing out the results.

Test case reporting

Test Days App: A web application for providing results from Test Days events. It uses ResultsDB as a backend.

Bug reporting

Blocker Bugs App: A web application to display current blocker bugs during our pre-release cycle.

Test update feedback

Fedora Gooey Karma: A graphical tool that makes it easy to report feedback for any updates-testing packages currently installed.

Fedora Easy Karma: A command line tool that makes it easy to report feedback for any updates-testing packages currently installed.

Mixed bag

Fedora QA tools: A git repository containing various small tools.

Statistics

Fedora QA statistics tools: Tools for generating statistics on user contributions. relval also has some functions for doing this. See Recognition for more details.

Infrastructure tools

These tools we use as our support infrastructure during QA tools development.

Bitbucket team repository: A list of repositories that are related to our tools and needs.

Phabricator: Our main issue and code review tracker for most of our projects.

@@ Line 1: / Line 1: @@
 {{header|qa}}
-__NOTOC__
+{{autolang|base=yes}}
-The [[QA]] Project develops tools to make testing easier and more reliable.  Most tools are written in Python 2.5.
+The [[QA]] team develops tools to make testing easier and more repeatable.  We have several projects ranging from some that are pretty simple and new developer friendly to complex and challenging stuff at the cutting edge of the field. Contact [[User:tflink|Tim Flink]] or the {{fplist|qa-devel}} mailing list if you're interesting in helping to build new tools for Fedora QA.
-= Test Automation =
+= QA tools =
-* [https://fedorahosted.org/beaker Beaker] is an idea for a fully automated test lab.
-* The [[Automated QA Testing Project]] is an outgrowth of FudCon F11.
-= Test Case Management =
+== Test Automation ==
-* [https://fedorahosted.org/nitrate Nitrate] is a upcoming test management framework for test planning and test execution tracking
-= Installation =
+; [[Taskotron]] : Taskotron is our current framework for automatically executing a number of Fedora related tests. Both Fedora infrastructure generic and Fedora packages specific tests should be supported. Package maintainers should have an easy way of providing tests for their packages. This framework obsoleted [[AutoQA]]. You can see [https://taskotron.fedoraproject.org/ Taskotron in action].
-* [https://fedorahosted.org/snake SNAKE] - Smart Network Automated Kickstart Environment
-* [https://fedorahosted.org/cobbler Cobbler/koan]
-* Anaconda diagnostics?
-* Various Kickstart tests
-= Bugzilla =
+; [[ResultsDB]] : A generic storage of (automated) test results. Designed to be lightweight, simple, scalable and fast. It's used by Taskotron.
-* [https://fedorahosted.org/python-bugzilla/ python-bugzilla]
-* Improvements to [https://bugzilla.redhat.com/ Bugzilla]
-= Infrastructure =
+; [https://bitbucket.org/fedoraqa/execdb ExecDB] : This database with a front-end allows us to track tasks which are scheduled, running and finished in Taskotron. It provides a simple way to navigate people (mostly admin and developer-type folks) into different corners of the Taskotron infrastructure - BuildBot job details, ResultsDB results, logs and artifacts directories. Read some design [https://phab.qadevel.cloud.fedoraproject.org/w/taskotron/planning/execdb_features/ concepts].
-* [https://fedorahosted.org/bodhi/ Bodhi]
-Developing Open Source QA tools is complex, challenging stuff at the cutting edge of the field. Contact [[WillWoods]] if you're up to the challenge of building new tools for Fedora QA.
+; [https://openqa.fedoraproject.org openQA] : We use this tool (developed by our friends at [https://www.suse.com/ SUSE], now with contributions from us too) to automate certain release test cases, especially installation with [[Anaconda]]. We develop [https://bitbucket.org/rajcze/openqa_fedora Fedora tests] for openQA, along with a [https://bitbucket.org/rajcze/openqa_fedora_tools scheduler and some other tools]. Also see the [https://os-autoinst.github.io/openQA/ upstream home page].
+; [http://beaker.fedoraproject.org Beaker] : Beaker is an up and coming fully automated test framework that provides bare metal hardware provisioning including console and power management, and job scheduling and execution. Learn more at [http://beaker-project.org/ Beaker homepage].
+== Release validation testing ==
+; [https://pagure.io/fedora-qa/python-wikitcms/ python-wikitcms], [https://pagure.io/fedora-qa/relval/ relval] and [https://pagure.io/fedora-qa/testdays/ testdays]: A Python library and CLI based upon it for performing various [[QA:Release_validation_test_plan|release validation testing]] tasks in the [[Wikitcms]] 'test management system' (the set of wiki pages in which release validation test results are stored). testdays can perform a few basic operations on [[Test_Days|Test Day]] pages. [https://pagure.io/fedora-qa/relvalconsumer/ relvalconsumer] is a [https://www.fedmsg.com Fedmsg] consumer which creates validation test events.
+; [https://pagure.io/fedora-qa/check-compose/ check-compose]: A script for running informational checks on a Fedora compose and printing or mailing out the results.
+== Test case reporting ==
+; [[QA:TestdayApp|Test Days App]] : A web application for providing results from [[QA/Test Days|Test Days]] events. It uses [[ResultsDB]] as a backend.
+== Bug reporting ==
+; [https://qa.fedoraproject.org/blockerbugs/ Blocker Bugs App] : A web application to display current [[QA:SOP blocker bug process|blocker bugs]] during our pre-release cycle.
+== Test update feedback ==
+; [[Fedora Gooey Karma]] : A graphical tool that makes it easy to report feedback for any updates-testing packages currently installed.
+; [[Fedora Easy Karma]] : A command line tool that makes it easy to report feedback for any updates-testing packages currently installed.
+== Mixed bag ==
+; [https://pagure.io/fedora-qa/qa-misc Fedora QA tools] : A git repository containing various small tools.
+== Statistics ==
+; [https://pagure.io/fedora-qa/qa-stats Fedora QA statistics tools] : Tools for generating statistics on user contributions. [https://pagure.io/fedora-qa/relval/ relval] also has some functions for doing this. See [[QA:Recognition|Recognition]] for more details.
+= Infrastructure tools =
+These tools we use as our support infrastructure during QA tools development.
+; [https://bitbucket.org/fedoraqa Bitbucket team repository] : A list of repositories that are related to our tools and needs.
+; [[QA/Phabricator|Phabricator]] : Our main issue and code review tracker for most of our projects.
+[[Category:QA]]

Search

QA:Tools: Difference between revisions