From FedoraProject

Continuous integration aims to ensure broken changes do not affect other developers, packagers, maintainers or users. Continuous delivery aims to ensure broken changes do not get delivered or released.

The Manifesto

Because there are several Continuous Integration efforts, we need to set the basic rules to make sure we’re all playing the same game. When we call a game “football” we need to agree on what that means.

Here is a Continuous Integration manifesto that helps describe the goals, terminology, and rules.

Tests locations

The tests to be executed are stored in the dist-git repositories. The tests are stored or wrapped along-side the spec files that describe the software sources. The tests are branched, maintained along with the package or module they pertain to.

Package or RPM specific tests are stored in that package's dist-git repository. Such package repositories will have a rpms/ prefix in their name. Other tests refer to larger sets of functionality, called modules and are stored in a dist-git repository with modules/ prefix in their name.

Test format

Tests may be written all sorts of different ways, but have to be exposed and invoked in a standard way. They are stored or wrapped in the tests/ directory in a dist-git repository. There is a detailed description of how this is done:

Gating

When a test fails, continuous integration will be able to prevent the broken package change from affecting other changes. That gating will happen in Bodhi. Packages in the core operating system, such as those in Fedora Atomic Host are most interesting for gating, as when they break they affect the most users.

While gating is already technically possible we want to improve the integration of the CI pipelines before we turn this on. This involves having a mechanism to re-trigger a test run to ensure that tests failing due to transient errors don't block updates more than they should. We are also dedicated to have a high number of tests passing the pipeline before enabling gating. The progress can be followed in fedora-ci/AtomicCI#2.


Opting-in

This effort focus on the packages that are part of the Fedora Atomic Host at first, if can prepare tests for your packages but if they are not part of the Fedora Atomic Host, these tests will not be run and their results not be considered for gating. This is to limit the set of packages considered and allow an iterative approach starting small and growing from there. We do aim to provide testing for more than just the Fedora Atomic Host packages but we needed to start somewhere.

If your packages are part of Fedora Atomic Host, we strongly encourage you to join this effort. The procedure to opt-in is fairly simple

  • read the documentation ( above) about the tests format and how to invoke them
  • add these tests in a tests/ folder in the dist-git repository of your packages

Simply adding the tests in a tests/ folder in the dist-git repository of your packages will be considered as a will to have these tests run and gate your packages.

If you have any questions about the tests or the gating, feel free to reach out to us (cf the Contact section below).


Notifications

Notifications about CI results will be available in two ways:

  • FMN has been adjusted to notify by default every packager when any step of the CI pipeline fails on one of the package they maintain. So if you are a kernel maintainer and a commit made to the kernel dist-git repository fails to compose an OSTree, FMN will notify you of it.
  • Bodhi will include the Fedora Atomic CI results in its update page, just as it already includes tests results from taskotron.

Testing pipeline

The pipeline is under development

Contact

To get involved: