From Fedora Project Wiki

< CI

(Add link to Pagure)
(Migrated to the new Fedora docs site)
 
(14 intermediate revisions by the same user not shown)
Line 1: Line 1:
= Introduction =
+
Moved to: https://docs.fedoraproject.org/en-US/ci/quick-start-guide/
 
 
Are you eager to try out how the Fedora CI tests work? Do you want
 
to get a quick hands-on experience without having to read too much
 
documentation? This quick introduction for the impatient will show
 
you a minimal set of steps to execute existing tests as well as
 
provide useful links to resources where you can learn more.
 
 
 
= First Steps =
 
 
 
Install the following essential packages on your system:
 
 
 
sudo dnf install fedpkg libselinux-python standard-test-roles
 
 
 
Use <code>fedpkg</code> to clone the package git repository. See
 
the [[Package_maintenance_guide|Package Maintenance Guide]] for
 
more info about the tool.
 
 
 
fedpkg clone -a bash
 
 
 
Tests are defined according to the
 
[[CI/Standard_Test_Interface|Standard Test Interface]] in the
 
<code>tests</code> directory:
 
 
 
cd bash/tests/
 
 
 
Test coverage to be executed together with the basic set of
 
metadata is described in the
 
[https://src.fedoraproject.org/rpms/bash/blob/master/f/tests/tests.yml tests.yml]
 
playbook. Use <code>ansible-playbook</code> to run all available
 
tests for the classic environment on the local host:
 
 
 
ansible-playbook --tags=classic tests.yml
 
 
 
From the ansible output you can directly see an overall summary of
 
the testing. If you see <code>failed=0</code> at the end of the
 
log then all tests passed:
 
 
 
localhost: ok=29 changed=11 unreachable=0 failed=0
 
 
 
For more detailed test results check the <code>test.log</code> and
 
other files in the <code>artifacts</code> directory:
 
 
 
vim artifacts/test.log
 
 
 
That's it! You just executed test coverage for the Bash.
 
 
 
= Test Subjects =
 
 
 
To execute tests against different test subjects we need to
 
prepare the environment. Let's store the detailed test results in
 
<code>/tmp/artifacts</code>, use dynamic inventory as defined by
 
the [[CI/Standard_Test_Roles|Standard Test Roles]] and download
 
the latest Atomic Host image.
 
 
 
export TEST_ARTIFACTS=/tmp/artifacts
 
export ANSIBLE_INVENTORY=/usr/share/ansible/inventory
 
curl -Lo /tmp/atomic.qcow2 https://getfedora.org/atomic_qcow2_latest
 
 
 
Now let's try to run tests against all supported test subjects.
 
 
 
== Classic ==
 
 
 
Run tests against classic rpms installed on the system:
 
 
 
export TEST_SUBJECTS=''
 
ansible-playbook --tags=classic tests.yml
 
 
 
== Container ==
 
 
 
Run tests in a docker container:
 
 
 
export TEST_SUBJECTS=docker:docker.io/library/fedora:latest
 
ansible-playbook --tags=container tests.yml
 
 
 
== Atomic ==
 
 
 
Run tests against the Atomic Host:
 
 
 
export TEST_SUBJECTS=/tmp/atomic.qcow2
 
ansible-playbook --tags=atomic tests.yml
 
 
 
= Adding Tests =
 
 
 
Unless you are maintainer of the package, who has direct commit
 
access, create a fork of the package git repository using the Fork
 
button in [https://src.fedoraproject.org/rpms/bash Pagure] web
 
interface and add your private fork as a new remote. Create a
 
branch for your new tests. For example:
 
 
 
git remote add fork ssh://psss@pkgs.fedoraproject.org/forks/psss/rpms/bash.git
 
git checkout -b tests
 
 
 
Create new test coverage under the <code>tests</code> directory,
 
update the <code>tests.yml</code> file accorgingly or create a new
 
one. You might also find suitable test coverage in the
 
[https://upstreamfirst.fedorainfracloud.org/ Upstream First] repo.
 
Run tests and verify they are stable and working fine in all
 
supported environments. Add files to git, commit and push:
 
 
 
git add tests.yml new-test
 
git commit -m "Add CI tests using the Standard Test Interface"
 
git push fork tests:tests
 
 
 
It is a good idea to include more details and links in the commit
 
message to make the pull request easier for review:
 
 
 
Add CI tests using the Standard Test Interface
 
 
Adding initial set of basic functionality tests for bash
 
according to the Standard Test Interface [1]. See Quick Start
 
Guide for brief introduction into how to run these tests [2].
 
 
[1] https://fedoraproject.org/wiki/CI/Standard_Test_Interface
 
[2] https://fedoraproject.org/wiki/CI/Quick_Start_Guide
 
 
 
Create a new pull request from your <code>tests</code> branch
 
against the master branch in the Pagure web interface. You might
 
want to include an additional info about the tests such as:
 
 
 
There are three tests available: smoke and func have been tested
 
across all environments (classic, container, atomic), login is
 
relevant for classic only (because of a missing dependency).
 
Please, merge the tests into all currently supported branches.
 
 
 
See temporary workaround for [[CI/Pull_Requests|Pull Requests]]
 
unless you are member of the Fedora packager group.
 

Latest revision as of 15:47, 18 March 2019