QA/Automated Installer Testing

These are the current plans for automated testing of the installer. The specific goal is this: perform daily, basic, fully-automatic installations of Rawhide, and report the results to a public web page.

Hardware

• One server machine with enough disk space for a local mirror of rawhide
  • Roughly 15G per arch. Ouch.
• One test machine per arch - i386, x86_64, and ppc.
  • ppc is almost optional.
  • KVM/QEMU/Xen could be used on the server machine to test in virtual guests.
  • Must be PXE-capable, and configured to boot PXE before the hard drive
• Some form of power control
  • This could be non-automated - "if the machine is dead, hit the power button".

Software

• cobbler and koan
  • See http://fedorahosted.org/cobbler/ for more details
  • Note that snake is still being considered but cobbler/koan seem to be much better.
  • pxe_just_once: 1 in /etc/cobbler/settings
• Rawhide sync script
  • http://jkeating.fedorapeople.org/sync-rawhide.py is an example script
  • Or you could just let cobbler handle it.
• logging client/server
  • jlaska maintains a cobbler git branch that adds automated remote logging of installs (and PPC support!): git://fedorapeople.org/~jlaska/cobbler.git.
  • Enhanced logging patchset has been submitted to cobbler upstream for inclusion in 1.5: https://fedorahosted.org/pipermail/cobbler/2008-December/001942.html

Overall design

1. Daily rawhide compose completes. We get signal.
2. Rawhide gets synced up.
3. Each client machine is instructed to run koan to install the new rawhide.
   • This might happen via func or some other mechanism.
4. A %pre script in the kickstart runs the logging client
   • Watch the following log files:
     • /tmp: anaconda.log, syslog, ks-script.log, anacdump.txt, modprobe.conf
     • /mnt/sysimage/root: install.log, upgrade.log
   • This is all handled by the 'anamon' code in jlaska's cobbler git tree.
5. If something fails during install
   • anacdump.txt appears, or no log data for 30min
   1. Use cobbler to change the PXE target to reinstall stable OS:
      • cobbler system edit --name=NAME --netboot-enabled=1
   2. Use power control to restart the machine
   3. Close log task with status FAIL
6. Otherwise: %post script in kickstart creates a service to run on firstboot
   • Send system startup logs (to be checked for errors)
   • Send signal to logging server to close task with status OK
7. Logging server checks over collected logs for errors
   • avc errors in /var/log/messages, FAILED in /var/log/boot.log, etc.
   • Change status to WARN if found.