QA/Automated Installer Testing

These are the current plans for automated testing of the installer. The specific goal is this: perform daily, basic, fully-automatic installations of Rawhide, and report the results to a public web page.

Hardware

 * One server machine with enough disk space for a local mirror of rawhide
 * Roughly 15G per arch. Ouch.
 * One test machine per arch - i386, x86_64, and ppc.
 * ppc is almost optional.
 * KVM/QEMU/Xen could be used on the server machine to test in virtual guests.
 * Must be PXE-capable, and configured to boot PXE before the hard drive
 * Some form of power control
 * This could be non-automated - "if the machine is dead, hit the power button".

Software

 * and
 * See http://fedorahosted.org/cobbler/ for more details
 * Note that is still being considered but cobbler/koan seem to be much better.
 * in
 * Rawhide sync script
 * http://jkeating.fedorapeople.org/sync-rawhide.py is an example script
 * Or you could just let cobbler handle it.
 * logging client/server
 * jlaska maintains a cobbler git branch that adds automated remote logging of installs (and PPC support!):.
 * Enhanced logging patchset has been submitted to cobbler upstream for inclusion in 1.5: https://fedorahosted.org/pipermail/cobbler/2008-December/001942.html

Overall design

 * 1) Daily rawhide compose completes. We get signal.
 * 2) Rawhide gets synced up.
 * 3) Each client machine is instructed to run koan to install the new rawhide.
 * 4) * This might happen via or some other mechanism.
 * 5) A   script in the kickstart runs the logging client
 * 6) * Watch the following log files:
 * 7) * This is all handled by the 'anamon' code in jlaska's cobbler git tree.
 * 8) If something fails during install
 * 9) *  appears, or no log data for 30min
 * 10) Use cobbler to change the PXE target to reinstall stable OS:
 * 11) Use power control to restart the machine
 * 12) Close log task with status FAIL
 * 13) Otherwise:   script in kickstart creates a service to run on firstboot
 * 14) * Send system startup logs (to be checked for errors)
 * 15) * Send signal to logging server to close task with status OK
 * 16) Logging server checks over collected logs for errors
 * 17) * avc errors in, FAILED in  , etc.
 * 18) * Change status to WARN if found.
 * 1) Logging server checks over collected logs for errors
 * 2) * avc errors in, FAILED in  , etc.
 * 3) * Change status to WARN if found.