From Fedora Project Wiki

Revision as of 21:25, 9 February 2009 by Jkeating (talk | contribs) (Triggers: clarify some of the composes.)

It is time that Fedora starts doing automated QA testing.

Many minds met during FUDCon F11 Boston 2009 in order to brain storm on automated testing.

We identified a number of "triggers" to react to in order to test things, a number of tests to perform, and a set of places to notify of the results. Here is a quick dump of these three things.

Triggers

  • cvs checkin
  • koji build
  • bodhi request
  • repo compose
  • install tree compose
  • manual (periodic)
  • iso compose
  • new test creation
  • post test
  • comps change

Tests

  • repository Sanity (sublist here)
  • package installation
  • profile testing
  • tree sanity
  • rpmdiff
  • build log sanity
  • source sanity
  • build root sanity
  • fails to build from source
  • make check
  • comps grammar


Notifiers

  • <package>-owner@fedoraproject.org
  • project webpage
  • fedora-test-list@redhat.com or fedora-devel-list@redhat.com

Also discussed were timeline targets. Next 3 weeks, next 3 months, next 3 quarters. We feel we can accomplish a number of the triggers and tests within the next 3 weeks, focusing more package and repo sanity. In the next 3 months we can likely add functional testing of installer via Lab in a Box. Over the next 3 quarters we hope to be able to expand functional testing to individual packages.