From Fedora Project Wiki

< Kadischi

Revision as of 16:26, 24 May 2008 by Ravidiip (talk | contribs) (1 revision(s))

Fedora Documentation
Kadischi FAQ Development Schedule

Kadischi Documentation

About Kadischi

What is Kadischi?

Kadischi is an alternative to Pilgrim for Fedora Core based Live CD generation. It takes a Fedora Core RPM repository as an input and outputs a live ISO9660 CD image.


How does Kadischi work?

Basically, Kadischi uses Anaconda to install the system in a temporary directory (specified in the configuration file) on user's hard drive. It then executes the collection of scripts (stored in /usr/local/share/kadischi/post_install_scripts) in order to modify the system to run successfully in read-only environment (CD). After all modifications are done, Kadischi creates an initrd image, then compresses the system tree (actually, it creates a compressed copy, and then removes the original one), and finally creates an ISO9660 CD image of the system.

For more details on the process, see "Under the bonnet" section.

Bug reports

Use Bugzilla for bug reports and feature requests. For details on the procedure, take a look at bug reports and feature requests .

Credits

Kadischi is heavily based on readonly-root package from Stateless-linux project.

Many design decisions are collective work of people from fedora-livecd list.

There are also some ideas taken from linux4all project.

The name for the project came from Elliot Lee.

Previous project maintainer was Darko Ilic (darko.ilic@gmail.com). Now The project was maintained by Contributors

Since the project was started as one of the Googles Summer of Code projects, some credits also goes to Google  ;)

Requirements

The following packages should be installed in order to run Kadischi:

  • anaconda
  • busybox-anaconda
  • syslinux
  • mkisofs
  • squashfs-tools
  • e2fsprogs
  • perl
  • gnome-python2-gtkhtml2

To install them, run 

yum install anaconda busybox-anaconda syslinux mkisofs squashfs-tools e2fsprogs usermode gnome-python2-gtkhtml2

If you want to compile Kadischi from CVS as source archive, the following packages should be installed:

  • kudzu-devel
  • intltool
  • glib2-devel
  • autoconf
  • automake

To install them, run 

yum install kudzu-devel intltool glib2-devel autoconf automake

Notes

No notes at this time.

Downloading

For now, the only way to download Kadischi is via anonymous CVS.

To do that, execute the following in a terminal (type enter when prompted for password): 

export CVSROOT=:pserver:anonymous@cvs.fedoraproject.org:/cvs/devel
cvs -z3 login
cvs -z3 co kadischi

If you don't have cvs client installed on the machine, you can install it running 

yum install cvs

for platforms other than Fedora Core Linux, please use some other method.

Installation

Go to the root directory of the source tree, and run ./autogen.sh. It will configure package for building. If you would like to pass some options to ./configure, you can pass them to ./autogen.sh and it will forward them to ./configure.

After that, you have to run make in order to build the application.

Run make install to install the application. Default location is /usr/local/share/kadischi for application files and /etc/kadischi for configuration files.

If you decide to install the CVS tree of Kadischi rather than build RPM packages, please remember to include package zlib in the transaction set for the LiveCD or LiveDVD root filesystem. libz.so.1 is needed for Kadischi to work properly. You can do this by modifying your comps.xml file and selecting zlib explicitly from the Anaconda package selection screen. Until kadischi and kadischi-tools makes it into the main repositories for Fedora, this will be required.

RPM building

Building the Kadischi RPM package for use with Fedora Core requires you to first set up an RPM building environment for your user, this includes installing rpmdevtools and running fedora-buildrpmtree like so: 

yum install rpmdevtools
fedora-buildrpmtree

This will create the neccessary directory tree associated with package building under any non-root user environment.

A kadischi.spec file is provided for building the RPM. To do this simply get the Kadischi CVS, and tar and bzip the kadischi CVS directory like so: 

tar -cf kadischi.tar kadischi
bzip2 -9 kadischi.tar

Provided you have followed the steps above or already have a buildsystem setup for your user and the required RPM building packages installed you can simply run: 

rpmbuild -ta kadischi.tar.bz2

which will start the RPM build.

After the RPM build process completes, the binary and/or source package RPMs should be available in ~/rpmbuild/RPMS/i386 ready to be installed.

Please be sure to put the kadischi-tools RPM in your repository if you are using a local disk or network repository, and include it in your comps.xml file for inclusion in the LiveCD or LiveDVD root filesystem. This pulls in the necessary libz.so.1 needed from the zlib package. Until kadischi and kadischi-tools makes it into the main repository for Fedora this will be required.

Uninstallation

Go to the root directory of the source tree and run make uninstall. Make sure there are no extra files added to the application directories (for example /etc/kadischi/kadischi.conf~).

If you've installed via RPM just run as root to uninstall the Kadischi package from your system: 

rpm -e kadischi


Using Kadischi

Basic instructions

The basic way to run Kadischi is to type the following in a terminal 


kadischi path-to-the-repository path-to-the-iso-image


where path to the repository can be a path on the local file system, but can also be a path to ftp, http, a stock Fedora Core DVD, or some other type of repository like a directory on the HDD.

Example: 


kadischi /tmp/fc6 /tmp/fedora-live.iso


where /tmp/fc6 is our repository. For instructions on creating a local repository, consult "How to build a repository" section from Appendix.

Example 2: 


kadischi http://download.fedora.redhat.com/pub/fedora/linux/core/6/i386/os /tmp/fedora-live.iso

Example 3: (using mirror site) 

kadischi http://mirrors.kernel.org/fedora/core/6/i386/os/ /tmp/fedora-live.iso


for installing from http repository. For details on available installation methods, consult "Anaconda installation methods" in Appendix.

Command line options

Type kadischi --help for list of Kadischi command line options.

Running Kadischi automatically

Since Kadischi itself doesnt require any interaction, the only thing we have to do is to make anaconda run automatically. This can be done using kickstart files. If you supply anaconda with complete kickstart file, it will run without prompting (if ks file is incomplete, it will prompt you only for information unless using -C option, any information not specified Anaconda will prompt for).

If you want to run Kadischi with kickstart, use --kickstart command line option: 

kadischi /tmp/fc5 /tmp/fedora-live.iso --kickstart=/path/to/kickstart.cfg

Some examples of kickstart files that can be used are included in CVS or the package and can be found in /usr/local/share/kadischi/ks_examples after installation or kadischi/ks_examples/ before the installation.

Another thing that matters is bringing up the user interface. Even if supplied with complete ks file, anaconda will bring up the interface (GUI or TUI) which is something we dont want here. Luckily, anaconda can be run in command line mode, where no interface is initialized. So, if you want to run Kadischi in the background, you would do something like: 

kadischi /tmp/fc5 /tmp/fedora-live.iso --kickstart=/path/to/kickstart.cfg -C

where -C stands for command line. You can also use --cmdline instead of -C.

Please note that in command line mode, no interaction with anaconda is possible, so if you have an incomplete kickstart file, anaconda will just print an error message to STDOUT and stop (it will not abort execution).

Using this two command line options you could, for example, create a cron job that will create fresh ISOs from rawhide repository every day.

Customizing Live CD system

I'll give you here some ideas on how to perform the customization of the live CD. Please note that there are probably many clever ways that I don't know about :)

  • Using kickstart file, and all the options it provides, including %pre and %post install scripts.
  • Adding scripts to
  • /usr/local/share/kadischi/post_install_scripts/ (cvs snapshots)
  • /usr/share/kadischi/post_install_scripts/ (rpms)

or editing some of them that are already there. Those scripts are executed right after the system is installed to chroot environment, but they are NOT executed in the chroot environment, they are executed from the parent system. Kadischi sorts all executables from post_install_scripts directory by the name, and executes them passing them the path to the freshly installed system as an argument.

  • Creating rpms with modifications, and adding them to the repository.

Tip: If you would like to customize your live CD, take a look at the Under the bonnet section of this document. Pay special attention to the scripts in post_install_scripts directory, and to the /usr/local/share/kadischi/movefiles.py. Those files are well documented, and reading just the first couple of lines would be helpful.

Tips and tricks

  • Get your self a rewritable CD, you're gonna need it :) (but don't forget the system will work slower when run from RW disc)
  • If you are using kickstart file, add --excludedocs to %packages (like this: %packages --excludedocs) in order to skip installing docs to /usr/local/doc (this can save you some space, depending on the number of packages)
  • include as many packages as you want (well, not more that 10GB), and write iso image to a DVD disc
  • If you want to experiment with LiveCD to HDD installs, create a post_install_script which symlinks all the utilities Anaconda uses during installation like mke2fs and mkswap to /usr/bin where installation media looks for these items. Boot the LiveCD or LiveDVD and commence an install with just "kadischi". It will detect you're running from LiveCD and start an installation. Report your progress via Bugzilla or by IRC.

Under the bonnet

How does Kadischi work? (in details)

In this section, I'll describe what happens from the moment you run kadischi from the command line, to the moment it writes "Finished" in it's log. If you just want to use Kadischi, you probably dont need to read this, but if you want to customize your build system or you want to understand how Kadischi works for any other reason, you should continue reading.

When you type kadischi in your command line, /usr/bin/kadischi is executed. It is just a simple bash script which sets PYTHONPATH variable to include Kadischi libraries, and executes /usr/local/share/kadischi/kadischi.py.

kadischi.py is top-level script as such is in charge of coordination between all other elements. It does this:

  • processes command line options and arguments
  • checks if everything looks OK (the number of given arguments, presence of destination file, UID, presence of required RPMs, config file)
  • sets values to variables (from the information it gets from the command line and from the config file)
  • creates subdirectory named 'livecd_build_no${number}' under the specified build directory; all temporary files for that build will be stored there
  • invokes anaconda; anaconda is run with --rootpath option and all other options specified on the command line (like --kickstart, --cmdline etc.). This is actually the most important part of the process. Anaconda installs the whole system in a directory on the local hard drive.
  • invokes scripts from /usr/local/share/kadischi/post_install_scripts passing them as an argument the path to the build tree. Those scripts are used to modify the system in order to work properly from the LiveCD
  • invokes /usr/local/share/kadischi/livecd-mkinitrd.sh which makes an initramfs or gzipped CPIO archive that will be used on the live CD system as initial userspace or what was formerly a initrd initial RAM disk. It provides the initial root filesystem.
  • invokes /usr/local/share/kadischi/movefiles.py which moves all the files and directories that should be writable once the live CD is booted to a single directory. That directory is then compressed in kadischi.tar.gz. During the booting of live CD, this archive is unpacked on tmpfs drive, and extracted files and directories are mounted (with mount --bind) to their original locations.
  • creates a compressed copy of the whole system, SquashFS using mksquashfs is used for compression.
  • copies some files that shouldn't be compressed (kernel, initrd etc.) from the uncompressed tree to the compressed one
  • removes the uncompressed tree
  • invokes /usr/local/share/kadischi/create-iso.sh which creates iso image out of the compressed system tree and places the ISO9660 image where you specify.
  • cleans up the build directory.

How does generated LiveCD work?

After insertion of the finished LiveCD or LiveDVD, the PC recognizes a bootable media. Grub or ISOLINUX is invoked depending on what you used as a bootloader for the LiveCD or LiveDVD. After selecting the kernel which to boot.. the initramfs and kernel are loaded, the initramfs uncompressed in RAM. The kernel invokes /init or /linuxrc in the initramfs uncompressed root filesystem. The /init or /linuxrc script is a NASH script and NASH is responsible for the bootstrap process in early userspace. Some things that happen at this time are:

  • /proc and /sys psuedo filesystems are mounted.
  • Initial device nodes are created in /dev such as /dev/{null,zero.systty,tty,console,tty[0-12] ,loop[0-7] }
  • Some internal NASH functions are performed, such as mkblkdevs and mkdmnode as well as loading of modules neccessary for standard Linux filesystems.. jbd.ko and ext3.ko.
  • All USB hub modules and SCSI bus modules are loaded along with SquashFS support modules and loop device modules.
  • We try to find the LiveCD, at this time we probe the busses for a valid device with a .livecd file in the root. find_live_cd.c is used to do this.
  • We mount the root filesystem which is the SquashFS image in the root of the LiveCD or LiveDVD.
  • We then run some NASH internals setuproot and busybox's switchroot to continue the boot process.


At this time /etc/rc.readonly in the compressed SquashFS filesystem is started with the parameter "linuxrc" to let the system know we are coming from initramfs and /init or /linuxrc. During this time we do some of the following:

  • Mount 2 directories, /livecd/{tmpfs,ramfs}
  • Extract our kadischi.tar.gz writable files to /livecd/tmpfs which is in RAM.
  • Bind mount some directories from / to /livecd/tmpfs for writeability.
  • Move /etc/rc.d on the compressed system to /livecd/ramfs for safe shutdown.
  • Run the requested runlevel scripts and startup files.

That covers most of the boot process of the LiveCD or LiveDVD. After this the system runs as normal Fedora Core systems would run after invoking services and other operations which a runlevel will require.

Design notes

No notes at this time.

Kadischi source tree

It's self-explanitory I hope, but it won't hurt me if I write a couple of lines here. I'll mention only directories, files are already documented (well, most of them).

kadischi/ bootsplash/ Contains files used in the stock Fedora Core releases for bootsplash images and memtest kernels. conf/ Holding all configuration files. After installation, these files are located in /etc/kadischi doc/ Kadischi documentation find_live_cd/ Sources for find-live-cd utility. find-live-cd is invoked by /linuxrc, and is in charge of finding the drive holding our LiveCD and mounting it ks_examples/ Some examples of kickstart files that can be used with Kadischi lib/ Functions and modules that are used by Kadischi linuxrc/ linuxrc directory which holds the script that will be used in LiveCD initramfs image livecd_generator/ This directory holds the main script, the one that is in charge of the whole process of LiveCD generation. There are also some scripts that are used during the process. man/ kadischi(1) and kadischi.conf(5) manual pages or any other manual documents. patches/ Patches that we made to anaconda to make it work with Kadischi. One day when we make sure theyre OK, they should go upstream po/ A directory where pot files should be placed. There is nothing there for now post_install_scripts/ A collection of scripts that are executed on the new system after Anaconda does it's work, in order to modify it to run properly from Live CD. Some of them can be tweaked in order to customize the system. rc/ Scripts that are used during the initialization of the Live CD system scanswap/ scanswap is utility that searches for swap partition on local hard drives, and enables it if it finds one

Appendix

How to build a repository

If you have Fedora Core installation CDs its a good idea to copy them to your hard drive and to perform installation from a harddrive repository since it is quick and easy, however any method FTP, HTTP or straight from a DVD will work. Heres the procedure:

  • create the root directory for your repository, for example /tmp/fc6 and create subdirectories /tmp/fc5/Fedora, /tmp/fc5/Fedora/RPMS, /tmp/fc5/Fedora/base like so:
mkdir -p /tmp/fc6/Fedora/{RPMS,base}
  • copy the contents of Fedora/base directory (from disc1) to /tmp/fc6/Fedora/base
  • copy the contents of Fedora/RPMS directory (from all 4 discs) to /tmp/fc6/Fedora/RPMS

Anaconda has been reworked a bit, Fedora Core 4 users should use anaconda-runtime utilities outlined on the Kadischi FAQ . Fedora Core 6 repository users should use createrepo. createrepo can be installed with 

yum install createrepo

To generate the repodata, if we have a repository in /pub/i386, cd in to /pub/i386 and use createrepo like so: 

createrepo -g Fedora/base/comps.xml .

And that's it. You can use this repository with Kadischi (path to the repository is /tmp/fc6).

Please note that FC4 repositories are not regular repositories, and you can't, for example, use them with yum.

Anaconda installation methods

  • method=nfs:<path> -- Use <path> for an NFS installation.
  • method=http://<path> -- Use <path> for an HTTP installation
  • method=ftp://<path> -- Use <path> for an FTP installation
  • method=hd://<dev>/<path> -- Use <path> on <dev> for a hard drive installation

(taken from [1] )

You can checkout /usr/share/doc/anaconda-${version}/install-methods.txt for further information.

buildstamp file

Anaconda uses buildstamp file to gather some information related to the distribution. Heres the example of the buildstamp file (for FC 4): 

200506062237.i386
Fedora Core
4
Fedora
http://bugzilla.redhat.com/bugzilla/

As you can see, there are 5 lines. The first line is something like the build ID, the second is product name, the third is product version, the forth is product path and the fifth is URL for bug reports.

The one that is very important to us is product path (the 4th one) because it is used in accessing the repository. If your repository is, for example, http://mirrors.kernel.org/fedora/core/5/i386/os/, anaconda will look for rpms in http://mirrors.kernel.org/fedora/core/5/i386/os/${productPath}/RPMS, so for the Fedora repository, the 4th line of the buildstamp should be "Fedora".

If you get an endless loop of "Reading package information... Unable to read header list. This may be due to a missing file or bad media. Press <return> to try again" from anaconda, it is very likely that there is something wrong with product path.

Anaconda looks for buildstamp file at /tmp/product/.buildstamp and /.buildstamp (in that order), so Kadischi puts it's buildstamp file to /tmp/product/.buildstamp so anaconda can find it. If the file /tmp/product/.buildstamp already exists, Kadischi will move it to a temporary location, put it's file to /tmp/product/.buildstamp, run anaconda, and when anaconda finishes the job, it will restore the original file.

Retrieved from "https://fedoraproject.org/w/index.php?title=Kadischi/Documentation&oldid=2673"