This page is still a draft, please consider it as such.
Jenkins @ Fedora-infra
What is Jenkins?
Jenkins is a continuous integration (CI) server. It runs the unit-tests of your project on a regular basis allowing early detection of problem in your code and making sure that your change are not too disruptive or do not break other part of your project.
If your project has unit-tests having CI is a nice thing. Ideally the tests should be ran after each commit and in a clean environment so that if additionnal dependencies have not been marked as such, the project will fail to run. Problem is, no-one runs the tests of his project after each change or has a clean environment in which to do so.
Jenkins is just one of a family of CI server which includes also buildbot, travis-ci and more.
What is Jenkins @ Fedora-infra?
The Fedora infrastructure now has its own dedicated private cloud system allowing to spin off/destroy easily virtual machin (VM) for any needs we could have.
Having easy throw-away VMs allows to build quickly Fedora image for any release desired.
Thus we decided to offer to our contributor the access to a Jenkins server which would allow to run the unit-tests of their project in Fedora and/or RHEL.
Now, you must be thinking: "This is awesome, but [ how can I add my project to jenkins?] and [ how can I configure jenkins the way I want?". No worries the answer to your question is just below.
Can I add my project to Jenkins?
The basic answer is yes, but there are of course some conditions to satisfy:
- MUST have a license thats acceptable for Fedora itself.
- MUST not include any items Fedora considers forbidden.
- MUST have a public source repo
If your project satisfies these conditions, keep reading ;-)
How can I add my project to Jenkins?
This is actually a very simple process, create a ticket on the fedora-infrastructure project on pagure with the following information:
- Subject: [Jenkins] project request: <name of your project>
- name of the project
- One line description of the project
- Contact person for this / admin for the project
- URL to the sources of the project
The URL is actually not needed but asked so that we can have a quick look at your project and see if (at least) license wise it follows Fedora's philosophy.
How can I use Jenkins?
To login into Jenkins, you will have to login using [ FAS] and via [ FAS-OpenId]
Jenkins is configured with a per project access policy. This means that when the people processing the Jenkins request on the fedora-infrastructure trac create the project, they will give admin right to that project to whoever is specified in the ticket.
To have the rights to run a project on Jenkins, you need the appropriate ACL that only a the project admin can provide you (or one of the Jenkins admin in case of problem).
How to configure Jenkins?
If you are an admin or have the appropriate rights, you can give rights to your co-workers to run builds, clean the workspace, destroy the project.
To do so:
- Login in Jenkins
- Go to your project
- Go to configure ( http://jenkins.fedorainfracloud.org/job/<project>/configure )
- The checkbox for "Enable project-based security" should already be checked.
- There you have the ACL matrix where you can add a new user (using his/her FAS name) and give/remove ACL to these users
Build my project
Jenkins aims at running your unit-tests in a clean environment, meaning you have to take care of specifying the dependencies (or in some case, ask fedora-infrastructure to add them to the builders).
To configure the build of your project
- login in Jenkins
- go to your project
- go to configure ( http://jenkins.fedorainfracloud.org/job/<project>/configure )
- There you can set:
- where are the sources (git/subversion/bazaar/mercurial)
- which branch to build (leave to '*' to build all the branches)
- when the project should be run (every hour, every day or just when there are changes)
- where the project can be run (see "Restrict where this project can be run") to restrict building on for example Fedora builders only)
- how the project should be run.
- if you are using python you want to make the build in a virtualenv which will provide you always a clean environment
- if you are using java, you might consider using maven to do the dependency management
Example of the shell script used to run the fedocal unit-tests:
rm -rf fedocalenv virtualenv fedocalenv source fedocalenv/bin/activate ## we need kitchen before we can do the list pip install kitchen ## python-fedora fails the first time, so we try twice pip install -r requirements.txt || pip install -r requirements.txt pip install nose --upgrade ## Needed within the venv hash -r ## Reload where the nosetests app is (within the venv) - see
which nosetestswith and without pip install nosexcover FEDOCAL_CONFIG=../tests/fedocal_test.cfg PYTHONPATH=fedocal nosetests --with-xcoverage --cover-erase --cover-package=fedocal PYTHONPATH=fedocal pylint -f parseable fedocal | tee pylint.out pep8 fedocal/*.py fedocal/*/*.py | tee pep8.out deactivate
There are multiple builders in the Jenkins infrastructure. You may need to restrict your jobs to only one kind. The labels are documented in git here:
Running jobs requiring password-protected SSH keys
Store your password in the Jenkins configuration script:
set +x export SSHPASS=noOneLikesPWs set -x bash my_build_script
The SSHPASS environment variable is consumed by sshpass.
In this example, the actual build script is stored, as recommended, with your code and not in Jenkins.
Determine if you need only a few SSH commands or a lot of them:
Only a few SSH commands
Just provide the password for each one in your build script.
sshpass -P assphrase -v -e ssh -i /path/to/your/private/key email@example.com command sshpass -P assphrase -v -e scp -i /path/to/your/private/key local_content firstname.lastname@example.org:/destination
Lots of SSH commands (or git and the like)
In your build script, start an ssh-agent and load the key. Then run your commands:
eval "$(ssh-agent -s)" sshpass -P assphrase -v -e ssh-add /path/to/your/private/key ...
A word about Host Keys
You may need to use
` on your first SSH connection to save the hostkey.
This is incomplete!
If you are missing a plugin, an information on how to set-up your project, or if you just have any question regarding our Jenkins, feel free to drop-by #fedora-admin on [ IRC] or create a ticket in the [ fedora-infrastructure trac] or send an email to the [ infrastructure mailing-list].