From Fedora Project Wiki

(Create page, brain dump from fudcon.)
 
(Link to some things)
Line 1: Line 1:
 
It is time that Fedora starts doing automated QA testing.
 
It is time that Fedora starts doing automated QA testing.
  
Many minds met during FUDCon F11 Boston 2009 in order to brain storm on automated testing. (link to video here)
+
Many minds met during FUDCon F11 Boston 2009 in order to brain storm on [http://alt.fedoraproject.org/pub/alt/videos/2009/FUDConF11/qa.ogg automated testing].
  
 
We identified a number of "triggers" to react to in order to test things, a number of tests to perform, and a set of places to notify of the results.  Here is a quick dump of these three things.
 
We identified a number of "triggers" to react to in order to test things, a number of tests to perform, and a set of places to notify of the results.  Here is a quick dump of these three things.
Line 36: Line 36:
 
* fedora-test-list@redhat.com or fedora-devel-list@redhat.com
 
* fedora-test-list@redhat.com or fedora-devel-list@redhat.com
  
Also discussed were timeline targets.  Next 3 weeks, next 3 months, next 3 quarters.  We feel we can accomplish a number of the triggers and tests within the next 3 weeks, focusing more package and repo sanity.  In the next 3 months we can likely add functional testing of installer via Lab in a Box (link here).  Over the next 3 quarters we hope to be able to expand functional testing to individual packages.
+
Also discussed were timeline targets.  Next 3 weeks, next 3 months, next 3 quarters.  We feel we can accomplish a number of the triggers and tests within the next 3 weeks, focusing more package and repo sanity.  In the next 3 months we can likely add functional testing of installer via [http://jlaska.livejournal.com/3230.html Lab] [http://jlaska.livejournal.com/3696.html in a] [http://jlaska.livejournal.com/3910.html Box].  Over the next 3 quarters we hope to be able to expand functional testing to individual packages.

Revision as of 23:24, 4 February 2009

It is time that Fedora starts doing automated QA testing.

Many minds met during FUDCon F11 Boston 2009 in order to brain storm on automated testing.

We identified a number of "triggers" to react to in order to test things, a number of tests to perform, and a set of places to notify of the results. Here is a quick dump of these three things.

Triggers

  • cvs checkin
  • koji build
  • bodhi request
  • rawhide compose
  • manual (periodic)
  • iso compose
  • new test creation
  • post test
  • comps change


Tests

  • repository Sanity (sublist here)
  • package installation
  • profile testing
  • tree sanity
  • rpmdiff
  • build log sanity
  • source sanity
  • build root sanity
  • fails to build from source
  • make check
  • comps grammar


Notifiers

  • <package>-owner@fedoraproject.org
  • project webpage
  • fedora-test-list@redhat.com or fedora-devel-list@redhat.com

Also discussed were timeline targets. Next 3 weeks, next 3 months, next 3 quarters. We feel we can accomplish a number of the triggers and tests within the next 3 weeks, focusing more package and repo sanity. In the next 3 months we can likely add functional testing of installer via Lab in a Box. Over the next 3 quarters we hope to be able to expand functional testing to individual packages.