From Fedora Project Wiki

< CI

(Added links to tests/ruby repository)
(Migrated to the new Fedora docs site)
 
(8 intermediate revisions by 2 users not shown)
Line 1: Line 1:
= Motivation =
+
Moved to: https://docs.fedoraproject.org/en-US/ci/share-test-code/
 
 
In order to make the CI workflow reliable and efficient it is
 
crucial to keep the test coverage in a good shape at all times.
 
Sharing test code between several packages (even within multiple
 
branches of the same package) may significantly help to:
 
 
 
* Prevent test code duplication
 
* Minimize test maintenance
 
* Catch incompatibilities early
 
 
 
In general, tests define how the software works and the basic
 
functionality of many packages doesn’t change that often. We try
 
hard to keep the backward compatibility where possible. Thus it
 
seems natural that, for such components, tests guarding the spec
 
could change at a slower pace than the distribution branches.
 
 
 
See the whole [https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org/thread/55U6V6UHA54MJLD2F6JF46EOLMVRUAE7/ ci-list discussion] for some more context.
 
 
 
= Implementation =
 
 
 
Store test code in your preferred repository and reference the
 
tests from the dist-git yaml file. There is also a special
 
<code>tests</code> namespace dedicated for storing Fedora CI
 
integration tests:
 
 
 
* https://src.fedoraproject.org/projects/tests/*
 
 
 
Some of the [[CI/Standard_Test_Roles|Standard Test Roles]]
 
(currently beakerlib and rhts) support fetching test code from
 
remote repositories directly in their config in this way:
 
 
 
- role: standard-test-beakerlib
 
  repositories:
 
  - repo: "https://src.fedoraproject.org/tests/shell.git"
 
    dest: "shell"
 
 
 
= Examples =
 
 
 
Here are some real-life examples where sharing test code can
 
increase long-term efficiency.
 
 
 
== Shell ==
 
 
 
There are several shells which implement the POSIX specification:
 
bash, ksh, mksh, zsh, dash. All of them share a significant amount
 
of test coverage and it does not make sense to commit & maintain
 
identical tests in five different repositories (+ possible
 
branches).
 
 
 
Shell tests repository:
 
 
 
* https://src.fedoraproject.org/tests/shell
 
 
 
Bash [https://src.fedoraproject.org/rpms/bash/blob/master/f/tests/tests.yml tests.yml]:
 
 
 
- hosts: localhost
 
  roles:
 
  - role: standard-test-beakerlib
 
    tags:
 
    - classic
 
    repositories:
 
    - repo: "https://src.fedoraproject.org/tests/shell.git"
 
      dest: "shell"
 
    tests:
 
    - shell/func
 
    - shell/login
 
    - shell/smoke
 
    required_packages:
 
    - expect            # login requires expect
 
    - which            # smoke requires which
 
 
 
Ksh [https://src.fedoraproject.org/rpms/ksh/blob/master/f/tests/tests.yml tests.yml]:
 
 
 
- hosts: localhost
 
  roles:
 
  - role: standard-test-beakerlib
 
    tags:
 
    - classic
 
    repositories:
 
    - repo: "https://src.fedoraproject.org/tests/shell.git"
 
      dest: "shell"
 
    tests:
 
    - shell/func
 
    - shell/login
 
    - shell/smoke
 
    environment:
 
      PACKAGES: ksh
 
      SH_BIN: ksh
 
    required_packages:
 
    - ksh
 
    - expect            # login requires expect
 
    - which            # smoke requires which
 
 
 
== Ruby ==
 
 
 
Another example is Ruby: With about 80 packages related to Ruby on
 
Rails it would be useful and efficient to have a single place for
 
integration tests which verify that the framework is correctly
 
working after updating any of these packages. Conversely,
 
maintaining those tests in 80 repos would be a tedious task.
 
 
 
Currently the shared [[https://src.fedoraproject.org/tests/ruby|tests/ruby]] repository hosts these three
 
ruby integration tests:
 
 
 
Available integration tests:
 
* systemtap-static-probes-in-ruby - exercising ruby's systemtap api
 
* bundler-unit-test - run bundler's unit tests
 
* run-basic-rails-application - run a simple rails application
 
 
 
== SELinux ==
 
 
 
Several SELinux user space components are sharing test coverage in
 
a single [https://src.fedoraproject.org/tests/selinux.git selinux] test
 
repository:
 
 
 
* [https://src.fedoraproject.org/rpms/libsepol/blob/master/f/tests/tests.yml libsepol]
 
* [https://src.fedoraproject.org/rpms/libselinux/blob/master/f/tests/tests.yml libselinux]
 
* [https://src.fedoraproject.org/rpms/libsemanage/blob/master/f/tests/tests.yml libsemanage]
 
* [https://src.fedoraproject.org/rpms/libsepol/blob/master/f/tests/tests.yml policycoreutils]
 
 
 
= Start =
 
 
 
In order to create a new repository in the tests namespace file a
 
[https://pagure.io/fedora-infrastructure/new_issue new issue] in
 
the fedora infrastructure queue. Specify desired repository name,
 
main admin and a short description/justification. For example:
 
 
 
* https://pagure.io/fedora-infrastructure/issue/6699
 
 
 
Creating new test repositories will be soon supported directly in
 
the <code>fedpkg</code> tool as well:
 
 
 
* https://pagure.io/fedpkg/issue/176
 

Latest revision as of 15:55, 18 March 2019

Moved to: https://docs.fedoraproject.org/en-US/ci/share-test-code/