Architectures/ARM/RfsBuild

= Rfsbuild = A utility for creating Root File Systems. Tony Egan Jan-08

There's a  Quickstart  below for the impatient.  Also, all the download links are collected at the bottom of this document.

Introduction
is intended to help developers create Root File Systems for embedded systems. This initial release is aimed at the ARM architecture. It was tested using  emulating an  board and using packages from the Fedora ARM repositories.

Development Environment
There are two typical environments for embedded system development. One, a native environment where the compilation and build and sometimes even the source editing takes place on the target processor. This requires a target which is powerful enough in terms of speed and resources to support such development.

The other is a cross-development environment where a cross-compiler and build tools are run on a host (typically an x86 PC) to generate code which is then downloaded to the target system. This leverages the power of the PC for the development work and requires nothing from the target except the resources to run the final application.

can be used in either of these development environments.

Components
The main  utility is a python program which can run on any architecture that supports Python 2.4 and. The main tools used by are   and QEMU. which is also written in Python is a package management utility used by some major Linux distributions, most notably Fedora. depends on a number of other utilities and libraries including,  ,  , etc.. In fact  requires a fairly capable linux environment in order to function.

QEMU is an emulator which runs on Linux and allows binary code for several different architectures to run on an Intel-based system. The first release of  was tested using   emulating an  system. This can support the Linux environment required by.

Operation
builds the target RFS from RPM packages. Obviously the RPMs must have been built for the target architecture. RPMs are available from many repositories on the Internet. The ARM RPMs used for testing were produced by the hard work of Lennert Buytenhek and others and were obtained here:

http://ftp.linux.org.uk/pub/linux/arm/fedora/pub/fedora/linux/core/6/arm/os/ http://ftp.linux.org.uk/pub/linux/arm/fedora/pub/fedora/linux/core/updates/6/arm http://ftp.linux.org.uk/pub/linux/arm/fedora/pub/fedora/linux/extras/6/arm
 * Fedora Core 6

http://ftp.linux.org.uk/pub/linux/arm/fedora/pub/fedora/linux/releases/8/Everything/arm/os/Packages http://ftp.linux.org.uk/pub/linux/arm/fedora/pub/fedora/linux/updates/8/arm
 * Fedora 8

You will also likely be developing your own RPMs containing your embedded application. You can make these available to  in a local repository. supports NFS access to local repositories saving you the trouble of setting up web access. This is discussed further below.

takes a list of packages and a list of repositories where the packages can be found. It passes these on to  which installs the RPM packages into the target RFS. takes care of all the dependency checking, downloading and installation of the packages. must execute on the target architecture in order to do this properly hence the need for a native system or an emulated system (QEMU) on which to execute.

Here is a sample invocation of  on a PC host:

rfsbuild --repo=c6,http://ftp.linux.org.uk/pub/linux/arm/fedora/pub/fedora/linux/core/6/arm/os/ --repo=c6u,http://ftp.linux.org.uk/pub/linux/arm/fedora/pub/fedora/linux/core/updates/6/arm --repo=c6e,http://ftp.linux.org.uk/pub/linux/arm/fedora/pub/fedora/linux/extras/6/arm --package=bash --package=nfs-utils --server=./startQEMU --rfspath=/newrfs --initrd=0 --verbosity=2

The repositories, specified by, are the Fedora ARM repositories mentioned earlier. Each one is prefixed by an arbitrary unique name, e.g. c6e, which  will use in its reporting. RPMs are specified by. You can specify as many repositories and packages as you like. Keep in mind that dependencies will be handled automatically. So when you specify,   will check its dependencies and install all the other packages required by bash. More on this later.

The  parameter specifies a script which starts the qemu-system-arm emulator. This is a bash script in this release, but it could be any type of executable. There are some requirements for how it invokes QEMU which we will discuss below. doesn't check the return from this script, but it may in future revisions.

uses NFS mounted directories to share information between the PC host and QEMU. The  parameter must specify an exported directory which can be mounted via NFS from QEMU. The new RFS will be built in a unique subdirectory of this directory, (e.g. ).  Existing RFS sub-directories will not be deleted or overwritten by.

The  parameter specifies that an init ramdisk image should be created from the new RFS. It will appear at for the example above. The zero means that  will automatically set the size of the initrd.

The  parameter tries to reassure the user that   is actually doing something.

When it starts,  first prints some informative messages then it copies itself to the directory specified by   and creates a run script  which will be used by QEMU to start   in the emulated environment. Rfsb is created in rfspath. then invokes the script specified by.

The program specified by  is responsible for starting QEMU. The  script provided tries to do this in a general way so that it can easily be adapted to different setups. Most of the parameters affecting the networking are specified at the top of the file. You need to edit this to suit your network. By the way the  script uses. You should make sure you have a line like: egan ALL=(ALL) NOPASSWD: ALL to your /etc/sudoers file.

Installing Rfsbuild
Simply download the RPM to your x86 host and enter: sudo rpm -ivh rfsbuild-0.9.0-beta.noarch.rpm

The RPM installs  and some miscellaneous support files in

directory where you can modify them and where you will invoke.
 * Copy  and   from   to a work


 * Download the ARM kernel and  to the same work directory.
 * -- kernel used by QEMU
 * -- initrd used by QEMU

You may want to rename these and create links using the original names in your work directory. Then when you want to test a new initrd or a a new kernel you can just change the links and keep the originals. Keep in mind that  will look for the files "zImage" and "initrd.gz".


 * Create a directory for your new RFS and export it via NFS.

mkdir /newrfs

echo "/newrfs *(rw,async,no_root_squash)" >> /etc/exports

Starting QEMU
Please see the local Architectures/ARM/HowToQemu and Fabrice Bellard's QEMU site for details of how to set up QEMU to emulate ARM. I have attached an ARM kernel and initrd which are convenient for  development. It is assumed that your QEMU installation is able to run the kernel and initrd provided and is able to access the host via the network so that it can NFS mount directories. It must also have access to the repositories specified by the  command. I.e. it must be able to access the internet.

A key feature of  which must not be affected by any modifications is its method of passing the path of the  run script on the kernel command line. When QEMU starts running the kernel and initrd provided, the root user is automatically logged in and the  script extracts this information from .   then mounts the directory specified on the command line and invokes the  script. The  script then starts in the QEMU environment. It sounds more complicated than it is.

You can test  by invoking it directly on the x86 host without any parameters. QEMU should start, boot Linux and automatically logon root and run the  script. This should fail harmlessly and leave you in QEMU at the Linux shell prompt.

To exit QEMU you can enter [Ctl-a]  [c]  to get to the QEMU monitor and then [q]  to quit. If this fails you can always open another shell on the host and enter: killall -9 qemu-system-arm To test NFS from QEMU, make sure NFS is enabled on the host and /newrfs is exported. Then invoke: ./startQEMU ip:/newrfs/abcde Where ip is the IP address of your x86 host. QEMU should start, boot Linux and automatically logon root and run the  script. The script should extract the NFS path from the kernel command line and mount the directory. The script should then fail harmlessly and leave you in QEMU Linux at the shell prompt. This is assuming there is no script in.

Then you can check that the NFS mount is working in QEMU : df -a /

If NFS is not working it is likely that QEMU networking is not set up properly or NFSD on the host is not successfully exporting.

Kill QEMU again.

At this point you should have  & NFS working. If not, you need to do some troubleshooting before continuing.

Local Repositories
At some point you will probably want to create your own RPMs in a repository on a local machine. Once you have created the RPMS just copy them to a directory on your host, e.g.  and then enter: createrepo /repo

You can then follow the procedure for making this repository accessible via http so that it can be accessed from anywhere. supports a simpler alternative by using a naming convention to specify local repositories which are accessible via NFS. On the host ensure that the repository directory is exported via NFS. Then the  parameter in the   command will look like: --repo=nfs10.0.0.105,file:///repo

The three letters " " are required, followed by the file ://  sees this name it will create an alias   and mount the repository via NFS prior to invoking.

Keystone Package
This idea comes from David Zeuthen's livecd-creator program, as does a lot of the code in. We saw above how to specify packages when invoking . This can get a bit unwieldy when there are a lot of packages. It would be nice if we could specify one package which in turn specified all the packages we want to install. That is the first function of a Keystone Package.

The other issue we haven't discussed is the ever present need to fix, tweak, modify and customize the RFS after the packages have been installed. RPMs already support post-install scripts but an embedded application often requires different modifications than the desktop application most RPMs are designed for. Different target devices usually require different post install modifications to the RFS. This is also handled by the keystone package.

A Keystone Package is an RPM which encapsulates a set of requirements for a particular RFS. So, you could have a Keystone Package for your router and another Keystone Package for your NAS device and another for your music server. Each would specify the specific set of packages and modifications required for each of those unique devices.

Keystone Packages take advantage of the version control, distribution and install mechanisms already in place for RPMs. You just give a user the appropriate version Keystone Package for his device and it encapsulates all the information needed to generate the RFS for that device perfectly. You don't even have to give him the package. If it is available in a repository on the Internet he just needs to specify it as a parameter to, and it will automatically be downloaded from the repository and installed.

The mechanism is very simple. A Keystone Package contains one or more special  scripts which are installed in a dedicated directory, . The package can contain miscellaneous other files also as needed just like any RPM but the   scripts are special.

During execution  first invokes   to install the packages specified on the command line. performs its normal dependency checking so more packages than those explicitly specified using  arguments may be installed at this time.

When this is finished  invokes any   scripts it finds in  with the parameter. See the script included in the  package. If there are multiple  scripts they will be invoked in alphabetical order. This call to the conf scripts will return a set of additional packages to be installed. invokes  a second time to install these. Again,  checks dependencies and installs all required packages.

When the second run of  is complete   invokes the scripts in  again but this time with the parameter. This causes the commands listed in the section of the conf scripts to be executed in a chroot context using the new RFS as root. I.e the new RFS appears to be the root directory while these  commands are executed. So any command specified in the  section must be available in the new RFS. This is where fixes, tweaks and modifications to the RFS are performed.

The sample Keystone Package included in the  RPM, uses the conf script . This Keystone Package was used to build the initrd supplied here for use with. So, to recreate the RFS and initrd you would simply need to invoke as follows:

rfsbuild --repo=nfs10.0.0.105,file:///repo --repo=f8,http://ftp.linux.org.uk/pub/linux/arm/fedora/pub/fedora/linux/releases/8/Everything/arm/os/Packages --repo=f8u,http://ftp.linux.org.uk/pub/linux/arm/fedora/pub/fedora/linux/updates/8/arm --package=fedora-rfsbuild-mini-glibc --package=findutils --server=./startQEMU --rfspath=/newrfs --initrd=0 --verbosity=2

This assumes you have put fedora-rfsbuild-mini-glibc.rpm in your local NFS exported repository 10.0.0.105:/repo.

-

Quickstart
For those who like to live dangerously.... :-)

ARM Host
If you have an ARM host that is already running Linux and supports python and  then you can simply install   on that machine and run it locally to build ARM RFSs. E.g.:

rfsbuild --repo=/repo --repo=c6,http://fedora-arm.wantstofly.org/pub/fedora/linux/core/6/arm/os --repo=c6u,http://fedora-arm.wantstofly.org/pub/fedora/linux/core/updates/6/arm --repo=c6e,http://fedora-arm.wantstofly.org/pub/fedora/linux/extras/6/arm --package=fedora-rfsbuild-mini-glibc --rfspath=/newrfs --initrd=0 --verbosity=2

Here the  directory is a local repository which contains the keystone package. is where we want the new RFS to be built. We are using ARM packages from the fedora-arm repositories. Since the  parameter is absent the build will be performed locally.

X86 Host and ARM QEMU server
How to set up an x86 host and an ARM QEMU environment so that can be invoked on the host to initiate a build under QEMU.


 * Install  on the X86 host if you haven't already done so.  This will create a directory /usr/share/rfsbuild containing the following:
 * README
 * startQEMU -- support utility for starting QEMU
 * qemu-ifup -- network configuration used by QEMU
 * 10-fedora-rfsbuild-mini-glibc.conf -- sample conf script
 * fedora-rfsbuild-mini-glibc.rpm -- sample keystone rpm
 * fedora-rfsbuild-mini-glibc.spec -- sample spec file
 * startrfsb -- this is installed in the /root directory of QEMU ARM linux. It starts a build when QEMU boots Linux.
 * Copy startQEMU and qemu-ifup to a work directory where you can modify them and where you will invoke rfsbuild.
 * Download the ARM kernel and initrd to the same work directory.
 * zImage -- kernel used by QEMU
 * initrd.gz -- initrd used by QEMU
 * Install QEMU and get it working with startQEMU and the zImage and initrd.gz you downloaded. This includes setting up networking via the TUN/TAP interface.  You can find some help at    and.

AT this stage invoking  with no parameters should start QEMU, boot Linux and automatically logon root and try to start an RFS build. This should fail harmlessly and leave you in QEMU Linux at the shell prompt.

To exit QEMU you can enter Ctl-a c to get to the QEMU monitor and then q to quit. If this fails you can always open another shell on the host and enter:

killall -9 qemu-system-arm


 * Create the RFS directory:

mkdir /newrfs


 * Export /newrfs via NFS.

Check that it is accessible from QEMU.

./startQEMU 10.0.0.105:/newrfs/abcde

Replace 10.0.0.105 with the IP address of your PC. This should behave the same as described above except you should see 10.0.0.105:/newrfs mounted on /newrfs when you check under QEMU:

df -a /

If NFS is not working it is likely that QEMU networking is not set up properly or NFSD on the host is not successfully exporting.

Kill QEMU again.

At this point you should have  &   working. If not, you need to do some troubleshooting before continuing.


 * Finally, invoke a test run of :

rfsbuild --repo=f8,http://ftp.linux.org.uk/pub/linux/arm/fedora/pub/fedora/linux/releases/8/Everything/arm/os/Packages --repo=f8u,http://ftp.linux.org.uk/pub/linux/arm/fedora/pub/fedora/linux/updates/8/arm --package=setup --server=./startQEMU --rfspath=/newrfs --verbosity=2

This should create a skeleton RFS with only the  package installed.

Miscellaneous Options

 * --exclude is used to omit packages from package groups. , and consequently rfsbuild will not exclude a package required by dependencies.


 * --cache specifies a  cache to use.  It's not expected to be empty.


 * --cacheonly tells  to use cache only i.e.


 * --initramfs create an initramfs archive. Not working.


 * --runscript the name of the script that  creates in   for QEMU to run.  The default is


 * --label the suffix of the RFS build directory. The default is constructed from the date.  Not working correctly yet since QEMU is not initializing the date correctly.

For those of you (like me) who don't like repeatedly typing long command lines I've included a sample script in the RPM called. Simply invoke it as: ./go fc8

This will re-create the sample initrd.

By the way the samples are just that "samples" to show you how  works. You will see some harmless error messages as you run the sample commands given above. The sample  script performs some very crude reductions on the RFS. Clean up and more structured pruning are left as an exercise for the reader. Useful revisions to the scripts and suggestions for improvement will be welcomed on the Fedora ARM mailing list.

--

Downloads

 * rfsbuild-0.9.0-beta.noarch.rpm
 * zImage
 * initrd.gz