From Fedora Project Wiki

(Add place holder page)
 
(Initial base page)
Line 1: Line 1:
Zuul Based CI
= Zuul Based CI =


in progress.
== What is Zuul ==
 
Zuul is the CI and gating system form the OpenStack project. It is able to scale fine and handle by default features such as artifacts sharing between jobs and cross Git repositories testing. You can see Zuul in action here [https://zuul.opendev.org/t/openstack/status].
 
Below is a list of features proposed by Zuul and its companion Nodepool:
 
* Event-driven pipelines based on Code-Review or Pull-Request workflow: jobs can be triggered automatically when a PR is submitted, changed, approved, or when the repository is tagged.
* CI-as-code: jobs are defined as YAML + Ansible playbooks, pipeline definitions as YAML files. Zuul reads and loads those definitions directly from Git repositories.
* Support for jobs inheritance, jobs dependencies, jobs chaining (with artifacts sharing).
* Speculative testing of new jobs before merging: jobs will be run as they are submitted to make sure they behave as expected.
* Cross repositories dependencies: a jobs' workspace can include unmerged patches from other projects if specified
* Parallel job run, only capped by resources available or predefined quotas
* Automated jobs resources lifecycle management: resources like VMs or containers needed by a given job can be defined in-repository, spawned on demand at a job's start, and destroyed when the job is finished, or held for debugging
* Job resources support of OpenStack, OpenShift, K8S, Static nodes, AWS.
* Well-defined, reproducible job environments to eliminate flakiness
* Speculative testing before merging (gating): if several patches are about to land at the same time, they are tested on the repository's future state.
 
Until now, Zuul was only able to listen to Gerrit or Github events, a new upcoming driver [https://review.opendev.org/604404/] will allow Zuul to interface with Pagure as well. Pagure, Zuul and Nodepool could therefore combine into a very efficient CI/CD stack.
 
== Pagure PR tests via the Zuul ==
 
Firstly we have worked on a Zuul driver for Pagure that bring all the nice features of Zuul usage with any recent Pagure instances (Zuul was designed for CI and Gating of OpenStack projects). It only relies on the Pagure web hook and API system.
 
We are able to run jobs and report results on PRs opened on Pagure instances like https://src.fedoraproject.org or https://pagure.io.
 
To do so we have deployed a Zuul/Nodepool instance (From the Software Factory project https://www.softwarefactory-project.io/) here: https://fedora.softwarefactoryproject.io/zuul/
 
Below ate some use cases for the PR workflow
 
* Build package on PR: https://stg.pagure.io/python-mock-distgit/pull-request/1
* Build package on PR which depends on another PR and validate a BuildRequire deps is handled [https://fedora.softwarefactory-project.io/logs/1/1/34192085e0eeb917376e5b7c27210b9468f0f597/check/rawhide-rpm-build/36955dd/job-output.txt.gz#_2019-05-20_13_43_59_894900] (artifacts sharing): https://stg.pagure.io/python-redis-distgit/pull-request/1
* Build package on PR then run child jobs to validate package via the package included tests (standard test interface): https://stg.pagure.io/attr/pull-request/2. Here a negative test where the source is changed to trigger a failure [https://fedora.softwarefactory-project.io/logs/2/2/be13de0bd8d43212f58db7ae3d214dc514abb64a/check/rawhide-rpm-test/2ee9b76/job-output.txt.gz#_2019-05-21_12_42_11_951856]
* Build package on PR then validate STI included functional tests and RPM lint via two childs job https://stg.pagure.io/python-redis-distgit/pull-request/2. Note that this RPM build is done on Koji as a scratch build [https://fedora.softwarefactory-project.io/logs/2/2/7d41b28e4e8004f5249043145851457746dc8e32/check/rawhide-rpm-koji-scratch-build/6f51d45/job-output.txt.gz#_2019-06-13_09_21_20_457455].

Revision as of 09:53, 14 June 2019

Zuul Based CI

What is Zuul

Zuul is the CI and gating system form the OpenStack project. It is able to scale fine and handle by default features such as artifacts sharing between jobs and cross Git repositories testing. You can see Zuul in action here [1].

Below is a list of features proposed by Zuul and its companion Nodepool:

  • Event-driven pipelines based on Code-Review or Pull-Request workflow: jobs can be triggered automatically when a PR is submitted, changed, approved, or when the repository is tagged.
  • CI-as-code: jobs are defined as YAML + Ansible playbooks, pipeline definitions as YAML files. Zuul reads and loads those definitions directly from Git repositories.
  • Support for jobs inheritance, jobs dependencies, jobs chaining (with artifacts sharing).
  • Speculative testing of new jobs before merging: jobs will be run as they are submitted to make sure they behave as expected.
  • Cross repositories dependencies: a jobs' workspace can include unmerged patches from other projects if specified
  • Parallel job run, only capped by resources available or predefined quotas
  • Automated jobs resources lifecycle management: resources like VMs or containers needed by a given job can be defined in-repository, spawned on demand at a job's start, and destroyed when the job is finished, or held for debugging
  • Job resources support of OpenStack, OpenShift, K8S, Static nodes, AWS.
  • Well-defined, reproducible job environments to eliminate flakiness
  • Speculative testing before merging (gating): if several patches are about to land at the same time, they are tested on the repository's future state.

Until now, Zuul was only able to listen to Gerrit or Github events, a new upcoming driver [2] will allow Zuul to interface with Pagure as well. Pagure, Zuul and Nodepool could therefore combine into a very efficient CI/CD stack.

Pagure PR tests via the Zuul

Firstly we have worked on a Zuul driver for Pagure that bring all the nice features of Zuul usage with any recent Pagure instances (Zuul was designed for CI and Gating of OpenStack projects). It only relies on the Pagure web hook and API system.

We are able to run jobs and report results on PRs opened on Pagure instances like https://src.fedoraproject.org or https://pagure.io.

To do so we have deployed a Zuul/Nodepool instance (From the Software Factory project https://www.softwarefactory-project.io/) here: https://fedora.softwarefactoryproject.io/zuul/

Below ate some use cases for the PR workflow