m (Minor Correction: 'make mockbuild' --> 'fedpkg mockbuild')
|Line 58:||Line 58:|
=== Using Mock inside your git sandbox ===
=== Using Mock inside your git sandbox ===
You only need to type '
You only need to type 'mockbuild' to start a mock build. The used architecture depends
on the directory where you start the mock build.
on the directory where you start the mock build.
Revision as of 11:21, 4 March 2013
This page is a collection of bits people have found useful when working with mock.
- 1 What is Mock?
- 2 How do I set up Mock?
- 3 How do I use Mock?
- 4 Building packages that depend on packages not in a repository
- 5 Caching in mock 0.8.x and later
- 6 Using Squid to Speed Up Mock package downloads
- 7 Using tmpfs to avoid disk access
- 8 Using mock under SELinux
- 9 Using mock as a chroot sandbox tool
What is Mock?
Mock takes a srpm and builds it in a chroot. This ensures that your BuildRequires lines are correct, that there are no missing dependencies, and that it builds cleanly. The Mock project page has more information on Mock.
How do I set up Mock?
First, install the mock package, which is available from Fedora Package Collection.
When using mock, you can use the "-r" option to select a particular test configuration. If there's one configuration that system users will normally use, you can set that as the default:
# cd /etc/mock # ln -s --force SOMECONFIG.cfg default.cfg
(An example of SOMECONFIG.cfg is "fedora-14-x86_64.cfg"; see the /etc/mock directory for the existing mock configurations.)
Setting up local mirror
If you want to set up a local mirror, refer MockSetupUsingLocalMirror
If you're behind a proxy, add the proxy info to the desired cfg file in /etc/mock in the same format that you would add it to your yum.conf. See man yum.conf for details on how to do it.
How do I use Mock?
Mock accepts a number of commands. Mock accepts commands in old-style format (e.g., "rebuild" or "init" without a leading "--"), but these are deprecated; use commands with a leading "--" instead.
Using Mock outside your git sandbox
Add your user name to the mock group
usermod -a -G mock myusername
Create your srpm using 'rpmbuild -bs'. Then change to the directory where your srpm was created.
Now you can start mock with
mock -r <configfile> --rebuild package-1.2-3.src.rpm
where <configfile> is the name of a configuration from
/etc/mock/ (do not include the
.cfg extension). Passing -r is not necessary if you have set a default config file via a symlink as shown above. --rebuild is the default option. So the following would work equally well
The --verbose option is useful if you want more output to be shown for debugging purposes.
If using mock version older than 0.8.8 or on a system with python 2.4, and building i386 packages on x86_64, prepend
setarch i386 to the mock command line:
setarch i386 mock -r <configfile> --rebuild package-1.2-3.src.rpm
Newer versions of mock no longer need the setarch command, although it does not hurt anything if it is there.
Using Mock inside your git sandbox
You only need to type 'fedpkg mockbuild' to start a mock build. The used architecture depends on the directory where you start the mock build.
Building packages that depend on packages not in a repository
If you're building some package P that depends on another package Q, and Q is not in a repository, you can still use mock. Here's one way to do that.
First, initialize the mock repository:
mock -r MOCK_CONFIG --init
(An example of MOCK_CONFIG is "fedora-14-x86_64"; see the /etc/mock directory for the existing mock configurations.)
Install the packages you need to build the program (named in BuildRequires) from the yum repositories and/or local RPMs. In practice, you may need to do a number of these (for the different package dependencies); you can list multiple packages on the same line:
mock -r MOCK_CONFIG --install PACKAGE_NAME_OR_PATH_TO_RPM
Current versions of mock should permit you to now build from the source RPM. If that does not work then the "--copyin" method below should be used. The --no-clean option is necessary since by default the chroot is cleaned prior to building.
mock -r MOCK_CONFIG --no-clean /PATH/TO/SRPM
Copy in the source RPM into /tmp (we'll copy in and do a build inside the shell, to work around the checks that detect that the packages aren't in the repository):
mock -r MOCK_CONFIG --copyin /PATH/TO/SRPM /tmp
Shell into the mock environment and perform the build:
mock -r MOCK_CONFIG --shell cd rpmbuild --rebuild /tmp/SRPM_NAME
Caching in mock 0.8.x and later
Mock 0.8.x introduces three different types of caches: 1) root cache, 2) yum cache, and 3) ccache. These caches greatly speed up mock.
Look in /etc/mock/defaults.cfg for documentation on configuring each of these caches. You can configure the directory where they are stored, the maximum age of the caches, and (for ccache only) the max disk usage for the cache.
Starting with mock 0.5, mock can automatically cache the standard buildroot for each environment in a local tar file (/var/cache/mock/$CONFIG/root_cache/*). It then unpacks this tar file to populate the buildroot, rather than downloading the buildroot RPMs fresh each time. After unpacking the buildroot from the root cache, mock does a
yum update to make sure that the buildroot is up-to-date prior to installing additional BuildRequires for a package build.
The root cache is enabled by default in mock 0.8.x and later. To disable, see the documentation in /etc/mock/defaults.cfg. The root cache can be disabled globally or per-chroot. The root caches are automatically deleted and recreated every 15 days to prevent them from being too stale.
Keep in mind that the root cache can affect reproducibility of your build, especially in environments where the set of base RPMs is updated often (eg. rawhide). Disable the root cache in environments where build reproducibility is of utmost importance.
By default, yum stores downloaded RPMs in a directory under /var/cache/yum. The yum cache feature bind mounts the chroot /var/cache/yum to a common directory outside the chroot environment (such as /var/cache/mock/$CONFIG/yum_cache/) where it can be saved and re-used by subsequent builds. This ensures that yum does not need to download each RPM fresh over the network for every build. This feature is enabled by default.
The ccache tool is a utility that wraps calls to compilers such as 'gcc' and caches the output. When called to compile the same program a second time (with the same cmdline args, gcc-version, and header files), ccache will instead pull the cached version rather than running the compiler afresh.
Using Squid to Speed Up Mock package downloads
The cached buildroots trick above is much faster than using squid alone, but you can also use squid in addition to the cached buildroots, particularly if you are at the slow end of an Internet connection.
The root cache will contain the standard buildroot packages, but as it gets older, the
yum update after unpacking it may have to update several packages. In addition, some packages aren't in the standard buildroot, but are often needed (e.g. pkgconfig). These can be faster served from a squid cache than from downloading them from the Internet for each package build.
Install the squid package which is available from Fedora Package Collection.
Some items worth changing in
- Maximum Object Size. Normally, this is 4MB. However, some of the base packages like glibc can be larger than 16MB, and if you're building say Eclipse plugins, that's 60MB. So make it large.
maximum_object_size 409600 KB
- Cache Replacement Policy. Normally, lru is fine. But in our case, we want to keep the largest objects around longer, and just download the smaller objects if we can.
cache_replacement_policy heap LFUDA
- Cache Size. Normal size is 100MB. This isn't going to cut it. This needs to be at least 1GB, more if you regularly build for multiple Fedora releases and multiple architectures.
cache_dir ufs /var/spool/squid 1000 16 256
Then start squid, and configure it to start on startup.
To make the build user automatically use the squid cache, type:
echo export http_proxy=\"http://localhost:3128\" >> /home/build/.bash_profile
If you only want to use squid for particular mock configurations (perhaps you have local mirrors of some distributions and don't need squid for those),
you can, instead of setting the
http_proxy environment variable, set up the proxy configuration in the
yum.conf part of each mock configuration file:
config_opts['yum.conf'] = """ [main] ... proxy=http://localhost:3128/ ... """
Edit /etc/mock/*.cfg to not use mirror lists
Don't use the mirrorservice.org mirror; they prevent downstream caching via the HTTP header "Cache-Control: no-store".
If you're using squid, you don't want mock+squid to use a random mirror for downloading packages. Change the config files in /etc/mock/*.cfg to use a single fast-for-you mirror in a
baseurl entry, and don't use
mirrorlists. You can use this script for that.
Using tmpfs to avoid disk access
It's not clear that tmpfs is a huge win anymore, now that autocache works and is a standard feature.
When having enough RAM (1.5-2 GB), using a tmpfs can speedup the 'prep' phase significantly. With 1.5GB RAM on a Celeron 2.8 GHz, populating the buildroot will take 2-5s (seconds) on subsequent builds.
You can either mount /var/lib/mock as a tmpfs, or you can mount each of the build directories under /var/lib/mock as separate tmpfs file systems. The latter lets you use the autocache above and keeps the autocache files on disk, rather than in the tmpfs. If you erase and recreate the autocaches often, keeping them in tmpfs makes sense. If you don't recreate them often, then you should use per-build-directory tmpfs mounts instead.
Using mock under SELinux
Using SELinux in conjuction with chroot-ed environments such as mock presents a number of problems,
such as needing to replicate all default file contexts with the chroot directory prefix, which is an
administrative nightmare. The approach used by mock is very simple, though it only works with the
preload a dummy
libselinux that makes all child processes think that SELinux is disabled.
With mock itself running as an unconfined process, this generally works well.
Problems with SELinux memory protection
However, from FC5 onwards, the SELinux
unconfined_t domain is no longer fully unconfined by SELinux;
memory protection is implemented to protect most user processes against
a variety of possible exploits.
This presents at least two sets of problems when using mock in FC5:
- When building packages for legacy distributions , the build process can involve loading old shared libraries that do not have separate sections for, say, executable code and writable data. This leads to
execmoddenials when a process running in the
unconfined_tdomain tries to load such a library, unless the library is labelled
- When building packages using mono (the same might apply to java), the
monoprocess would normally run in its own less-constrained domain (
mono_t); when running under mock,
monoruns in the
unconfined_tdomain, which can result in
Whilst it is possible to turn off these checks using SELinux booleans, this is not a desirable thing to do for all processes.
Another solution would be to ensure that all files were labelled correctly in the chroot and that SELinux domain transitions ocurred when needed. However, this is not possible because processes running under mock are told by the dummy
libselinux that SELinux is disabled.
SELinux policy module for mock
The following solution to these issues creates an SELinux policy for mock that has two aspects:
- It runs
mockin its own domain (
mock_t) that is as unconstrained as any process that needs to run under it needs to be, which so far means that it allows
execmod, just like the
- It labels all files installed under
mocksets up its chroot environments) with the special context type
mock_var_lib_t, for which
execmodis allowed for processes in the
This solution relaxes SELinux protection sufficiently for mock to be able to work, without compromising the protection afforded to the normal unconfined domain.
To install the policy module:
- Create a directory to store local policy files, such as
/root/selinux.local, and change to that directory
- Download File:PackageMaintainers MockTricks mock.if, File:PackageMaintainers MockTricks mock.fc, and File:PackageMaintainers MockTricks mock.te to this directory
- Build and install the policy module as follows:
# make -f /usr/share/selinux/devel/Makefile Compliling targeted mock module /usr/bin/checkmodule: loading policy configuration from tmp/mock.tmp /usr/bin/checkmodule: policy configuration loaded /usr/bin/checkmodule: writing binary representation (version 5) to tmp/mock.mod Creating targeted mock.pp policy package rm tmp/mock.mod.fc tmp/mock.mod
If the policy module is loaded before the
mock package is installed,
/var/lib/mock will be labelled as
/usr/bin/mock will be labelled as
mock_exec_t at installation time. Otherwise, it's necessary to use
restorecon to fix the file contexts:
# restorecon -R /var/lib/mock /usr/bin/mock
Using mock as a chroot sandbox tool
Mock can be used to create chroots for testing things, not just building packages. Here is a quick howto:
- Create a config file that points to the repo(s) of your choice, where your test packages are
mock -r <config-name> --init mock -r <config-name> --install <your packages> mock -r <config-name> --shell
Why use mock to shell, why not chroot directly? Using mock to "shell" will allow mock to create the mountpoints you'll probably need inside the chroot. If you want to manage your mounts manually, do so by all means.