Are you eager to try out how the Fedora CI tests work? Do you want to get a quick hands-on experience without having to read too much documentation? This quick introduction for the impatient will show you a minimal set of steps to execute existing tests as well as provide useful links to resources where you can learn more.
Install the following essential packages on your system:
sudo dnf install fedpkg libselinux-python standard-test-roles
fedpkg to clone the package git repository. See
the Package Maintenance Guide for
more info about the tool.
fedpkg clone -a bash
Tests are defined according to the
Standard Test Interface in the
Test coverage to be executed together with the basic set of
metadata is described in the
ansible-playbook to run all available
tests for the classic environment on the local host:
ansible-playbook --tags=classic tests.yml
From the ansible output you can directly see an overall summary of
the testing. If you see
failed=0 at the end of the
log then all tests passed:
localhost: ok=29 changed=11 unreachable=0 failed=0
For more detailed test results check the
other files in the
That's it! You just executed test coverage for the Bash.
To execute tests against different test subjects we need to
prepare the environment. Let's store the detailed test results in
/tmp/artifacts, use dynamic inventory as defined by
the Standard Test Roles and download
the latest Atomic Host image.
export TEST_ARTIFACTS=/tmp/artifacts export ANSIBLE_INVENTORY=/usr/share/ansible/inventory curl -Lo /tmp/atomic.qcow2 https://getfedora.org/atomic_qcow2_latest
Now let's try to run tests against all supported test subjects.
Run tests against classic rpms installed on the system:
export TEST_SUBJECTS= ansible-playbook --tags=classic tests.yml
Run tests in a docker container:
export TEST_SUBJECTS=docker:docker.io/library/fedora:latest ansible-playbook --tags=container tests.yml
Run tests against the Atomic Host:
export TEST_SUBJECTS=/tmp/atomic.qcow2 ansible-playbook --tags=atomic tests.yml
Unless you are maintainer of the package, who has direct commit access, create a fork of the package git repository using the Fork button in Pagure web interface and add your private fork as a new remote. Create a branch for your new tests. For example:
git remote add fork ssh://email@example.com/forks/psss/rpms/bash.git git checkout -b tests
Create new test coverage under the
tests.yml file accorgingly or create a new
one. You might also find suitable test coverage in the
Upstream First repo.
Run tests and verify they are stable and working fine in all
supported environments. Add files to git, commit and push:
git add tests.yml new-test git commit -m "Add CI tests using the Standard Test Interface" git push fork tests:tests
It is a good idea to include more details and links in the commit message to make the pull request easier for review:
Add CI tests using the Standard Test Interface Adding initial set of basic functionality tests for bash according to the Standard Test Interface . See Quick Start Guide for brief introduction into how to run these tests .  https://fedoraproject.org/wiki/CI/Standard_Test_Interface  https://fedoraproject.org/wiki/CI/Quick_Start_Guide
Create a new pull request from your
against the master branch in the Pagure web interface. You might
want to include an additional info about the tests such as:
There are three tests available: smoke and func have been tested across all environments (classic, container, atomic), login is relevant for classic only (because of a missing dependency). Please, merge the tests into all currently supported branches.
See temporary workaround for Pull Requests unless you are member of the Fedora packager group.