Using the Koji build system
(Imported from MoinMoin)
m (1 revision(s))
Revision as of 16:37, 24 May 2008
Using Koji in Fedora
The Koji Build System is Fedora's buildsystem for Fedora 7 and beyond. Packagers use the koji client to request package builds and get information about the buildsystem.
Installing the Koji Cli
yum install fedora-packager
fedora-packager provides useful scripts to help maintain and setup your koji environment. There is a single point of entry for most operations. The command is called 'koji' and is included in the main koji package (koji is installed when you install fedora-packager). By default the koji tool authenticates to the central server using Kerberos. However ssl and username/password authentications are available. You will need to have a valid auth token to use many features. However, many of the read-only commands will work without authentication.
Initial Fedora setup
In order to interface with the koji server, maintainers will need to run
Existing users of plague, it will reference certs already existing. If you did not have plague before, it will get the server CA certs and tell you where to get your user cert
Each user on a system will need to run fedora-packager-setup if they wish to use koji to build Fedora packages. Each user has thier own certificates that authenticate them.
Koji uses three certificates:
- .fedora.cert (specific to the Fedora Maintainer) This cert is generated from this form . It should have been generated when you became maintainer. You may need to refresh it when it expires.
- .fedora-upload-ca.cert (The certificate for the Certificate Authority used to sign the user keys.) It can be downloaded from here .
fedora-packager-setupshould fetch it.
- .fedora-server-ca.cert (The certificate for the Certificate Authority used to sign the buildsystem's server keys.) It can be downloaded from here .
fedora-packager-setupshould fetch it.
All of these certificates should be in your homedir (~).
The global local client configuration file for koji is
/etc/koji.conf. You should not need to change this from the defaults for building Fedora packages, as running
fedora-packager-setup will create a set of config files in ~/.koji/ file for your user these will allow you to use the primary buildsystem as well as secondary arch build systems.
The web interface
The primary interface for viewing Koji data is a web application. It is available at  . Most of the interface is read-only, but if you are logged in (see below) and have sufficient privileges there are some actions that can be performed though the web. For example:
- Cancel a build
- Resubmit a failed task
- Setup a notification
Those with admin privileges will find additional actions, such as:
- Create/Edit/Delete a tag
- Create/Edit/Delete a target
- Enable/Disable a build host
The web site utilizes SSL authentication. In order to log in you will need a valid SSL certificate and your web browser will need to be configured to trust the SSL cert. Instructions on how to do this are printed when running fedora-packager-setup.sh.
With Koji you can setup a notification requests, to make sure you do not miss when a package you care about gets built. Login and scroll to the bottom of the page, there you should find a Add a notification link and a list of your configured notifications.
Building with make targets
For simple build requests, there is an alias in Makefile.common to request koji builds. This enables Fedora packagers to simply cd into the appropriate branch of a package (from a cvs checkout), and run:
This will trigger a build request for the branch. Easy!
Note that all build requests need to be done against tagged trees (run
make tag first).
You can do a scratch build with:
if you want to do a scratch build for a specific architecture, you can type:
<archs> can be a comma separated list of severals architectures.
finally is possible to specified a special build target in the form:
make scratch-build TARGET='<target>'
Sometimes you want to make sure than one build succeeded before launching the next one, for example when you want to rebuild a package against a just rebuilt dependency. In that case you can use a chain build with:
make chain-build CHAIN='libwidget libgizmo'
The current package is added to the end of the CHAIN list. Colons (:) can be used in the CHAIN parameter to define dependency groups. Packages in a single group will be built in parallel, and all packages in a group must build successfully and populate the repository before the next group will begin building. If no groups are defined, packages will be built sequentially.
If your package fails to build, you will see something like this:
420066 buildArch kernel-2.6.18-1.2739.10.9.el5.jjf.215394.2.src.rpm, ia64): open (build-1.example.com) -> FAILED: BuildrootError: error building package (arch ia64), mock exited with status 10
You can figure out why the build failed by looking at the log files. If there is a build.log, start there. Otherwise, look at init.log.
Logs can be found via the web interface in the Task pages for the failed task. Alternatively the koji client can be used to view the logs via the
watch-logs command. See the help output for more details.
Advanced use of Koji
We've tried to make Koji self-documenting wherever possible. The command line tool will print a list of valid commands and each command supports --help. For example:
$ koji help Koji commands are: build Build a package from source cancel-task Cancel a task help List available commands latest-build Print the latest rpms for a tag latest-pkg Print the latest builds for a tag [...]
$ koji build --help usage: koji build [options] tag URL (Specify the --help global option for a list of other help options) options: -h, --help show this help message and exit --skip-tag Do not attempt to tag package --scratch Perform a scratch build --nowait Don't wait on build [...]
Using Koji to control tasks
List only tasks requested by you:
koji list-tasks --mine
requeue an already-processed task: general syntax is: koji resubmit [options] taskID
koji resubmit 3
Building a Package with the command-line tool
Instead of using the make target, you can also directly use the command_line tool, koji. To build a package, the syntax is:
$ koji build <build target> <cvs URL>
$ koji build dist-f8 'cvs://cvs.example.com/cvs/dist?rpms/kernel/FC-7#kernel-2_6_20-1_2925_fc7'
The koji build command creates a build task in Koji. By default the tool will wait and print status updates until the build completes. You can override this with the --nowait option. This can also be used with the make command by setting an ENV variable:
KOJI_FLAGS="--nowait" make build
Sometime it is useful to be able to build a package against the buildroot but without actually including it in the release. This is called a scratch build. To create a scratch of latest CVS commit:
koji build --scratch dist-f8 'cvs://cvs.fedoraproject.org/cvs/pkgs?rpms/yum/devel#HEAD'
You can also scratch build source rpms.
koji build --scratch dist-f8 mycoolpackage-3.2-1.src.rpm
There are a few options to the build command. Here are some more detailed explanations of them:
--skip-tag Normally the package is tagged after the build completes. This option causes the tagging step to be skipped. The package will be in the system, but untagged (you can later tag it with the tag-pkg command) --scratch This makes the build into a scratch build. The build will not be imported into the db, it will just be built. The rpms will land under <topdir>/scratch. Scratch builds are not tracked and can never be tagged, but can be convenient for testing. Scratch builds are typically removed from the filesystem after one week. --nowait As stated above, this prevents the cli from waiting on the build task. --arch-override This option allows you to override the base set of arches to build for. This option is really only for testing during the beta period, but it may be retained for scratch builds in the future.
In Koji, it is sometimes necessary to distinguish between the a package in general, a specific build of a package, and the various rpm files created by a build. When precision is needed, these terms should be interpreted as follows:
Package: The name of a source rpm. This refers to the package in general and not any particular build or subpackage. For example: kernel, glibc, etc. Build: A particular build of a package. This refers to the entire build: all arches and subpackages. For example: kernel-2.6.9-34.EL, glibc-2.3.4-2.19. RPM: A particular rpm. A specific arch and subpackage of a build. For example: kernel-2.6.9-34.EL.x86_64, kernel-devel-2.6.9-34.EL.s390, glibc-2.3.4-2.19.i686, glibc-common-2.3.4-2.19.ia64
Tags and targets
Koji organizes packages using tags. In Koji a tag is roughly a collection of packages:
- Tags support inheritance
- Each tag has its own list of valid packages (inheritable)
- Package ownership can be set per-tag (inheritable)
- When you build you specify a target rather than a tag
A build target specifies where a package should be built and how it should be tagged afterwards. This allows target names to remain fixed as tags change through releases.
You can get a full list of build targets with the following command:
$ koji list-targets
You can see just a single target with the --name option:
$ koji list-targets --name dist-fc7 Name Buildroot Destination --------------------------------------------------------------------------------------------- dist-fc7 dist-fc7-build dist-fc7
This tells you a build for target dist-fc7 will use a buildroot with packages from the tag dist-fc7-build and tag the resulting packages as dist-fc7.
You can get a list of tags with the following command:
$ koji list-tags
As mentioned above, each tag has its own list of packages that may be placed in the tag. To see that list for a tag, use the list-pkgs command:
$ koji list-pkgs --tag dist-fc7 Package Tag Extra Arches Owner ----------------------- ----------------------- ---------------- ---------------- ElectricFence dist-fc6 pmachata GConf2 dist-fc6 rstrode lucene dist-fc6 dbhole lvm2 dist-fc6 lvm-team ImageMagick dist-fc6 nmurray m17n-db dist-fc6 majain m17n-lib dist-fc6 majain MAKEDEV dist-fc6 clumens [...]
The first column is the name of the package, the second tells you which tag the package entry has been inherited from, and the third tells you the owner of the package.
To see the latest builds for a tag, use the latest-pkg command:
$ koji latest-pkg --all dist-fc7 Build Tag Built by ---------------------------------------- -------------------- ---------------- ConsoleKit-0.1.0-5.fc7 dist-fc7 davidz ElectricFence-2.2.2-20.2.2 dist-fc6 jkeating GConf2-2.16.0-6.fc7 dist-fc7 mclasen ImageMagick-126.96.36.199-3.fc6.1 dist-fc6-updates nmurray MAKEDEV-3.23-1.2 dist-fc6 nalin MySQL-python-1.2.1_p2-2 dist-fc7 katzj NetworkManager-0.6.5-0.3.cvs20061025.fc7 dist-fc7 caillon ORBit2-2.14.6-1.fc7 dist-fc7 mclasen
The output gives you not only the latest builds, but which tag they have been inherited from and who built them.