From Fedora Project Wiki

< CI

(Clarify ansible output)
(Adjustments)
Line 27: Line 27:
 
Test coverage to be executed together with the basic set of
 
Test coverage to be executed together with the basic set of
 
metadata is described in the <code>tests.yml</code> playbook. Use
 
metadata is described in the <code>tests.yml</code> playbook. Use
<code>ansible-playbook</code> to run all available tests:
+
<code>ansible-playbook</code> to run all available tests for the
 +
classic environment on the local host.
  
  ansible-playbook tests.yml
+
  ansible-playbook --tags=classic tests.yml
  
 
From the ansible output you can directly see an overall summary of
 
From the ansible output you can directly see an overall summary of
Line 40: Line 41:
 
other files in the <code>artifacts</code> directory:
 
other files in the <code>artifacts</code> directory:
  
  vim artifacts/*
+
  vim artifacts/test.log
  
 
That's it! You just executed test coverage for the Bash.
 
That's it! You just executed test coverage for the Bash.
Line 77: Line 78:
  
 
  export TEST_SUBJECTS=/tmp/atomic.qcow2
 
  export TEST_SUBJECTS=/tmp/atomic.qcow2
  ansible-playbook --tags=container tests.yml
+
  ansible-playbook --tags=atomic tests.yml

Revision as of 12:30, 26 February 2018

Introduction

Are you eager to try out how the Fedora CI tests work? Do you want to get a quick hands-on experience without having to read too much documentation? This quick introduction for the impatient will show you a minimal set of steps to execute existing tests as well as provide useful links to resources where you can learn more.

First Steps

Install the following essential packages on your system:

sudo dnf install fedpkg libselinux-python standard-test-roles

Use fedpkg to clone the package git repository. See the Package Maintenance Guide for more info about the tool.

fedpkg clone -a bash

Tests are defined according to the Standard Test Interface in the tests directory:

cd bash/tests/

Test coverage to be executed together with the basic set of metadata is described in the tests.yml playbook. Use ansible-playbook to run all available tests for the classic environment on the local host.

ansible-playbook --tags=classic tests.yml

From the ansible output you can directly see an overall summary of the testing. If you see failed=0 at the end of the log then all tests passed:

localhost: ok=29 changed=11 unreachable=0 failed=0

For more detailed test results check the test.log and other files in the artifacts directory:

vim artifacts/test.log

That's it! You just executed test coverage for the Bash.

Test Subjects

To execute tests against different test subjects we need to prepare the environment. Let's store the detailed test results in /tmp/artifacts, use dynamic inventory as defined by the Standard Test Roles and download the latest Atomic Host image.

export TEST_ARTIFACTS=/tmp/artifacts
export ANSIBLE_INVENTORY=/usr/share/ansible/inventory
curl -Lo /tmp/atomic.qcow2 https://getfedora.org/atomic_qcow2_latest

Now let's try to run tests against all supported test subjects.

Classic

Run tests against classic rpms installed on the system:

export TEST_SUBJECTS=
ansible-playbook --tags=classic tests.yml

Container

Run tests in a docker container:

export TEST_SUBJECTS=docker:docker.io/library/fedora:27
ansible-playbook --tags=container tests.yml

Atomic

Run tests against the Atomic Host:

export TEST_SUBJECTS=/tmp/atomic.qcow2
ansible-playbook --tags=atomic tests.yml