|
|
(9 intermediate revisions by the same user not shown) |
Line 1: |
Line 1: |
| = Introduction =
| | Moved to: https://docs.fedoraproject.org/en-US/ci/quick-start-guide/ |
| | |
| Are you eager to try out how the Fedora CI tests work? Do you want to get a quick hands-on experience without having to read too much documentation? This quick introduction for the impatient will show you a minimal set of steps to execute existing tests as well as provide useful links to resources where you can learn more.
| |
| | |
| = First Steps =
| |
| | |
| Install the following essential packages on your system:
| |
| | |
| sudo dnf install fedpkg libselinux-python standard-test-roles
| |
| | |
| Use <code>fedpkg</code> to clone the package git repository. See the [[Package_maintenance_guide|Package Maintenance Guide]] for more info about the tool.
| |
| | |
| fedpkg clone -a bash
| |
| | |
| Tests are defined according to the [[CI/Standard_Test_Interface|Standard Test Interface]] in the <code>tests</code> directory:
| |
| | |
| cd bash/tests/
| |
| | |
| Test coverage to be executed together with the basic set of metadata is described in the [https://src.fedoraproject.org/rpms/bash/blob/master/f/tests/tests.yml tests.yml] playbook. Use <code>ansible-playbook</code> to run all available tests for the classic environment on the local host:
| |
| | |
| ansible-playbook --tags=classic tests.yml
| |
| | |
| From the ansible output you can directly see an overall summary of the testing. If you see <code>failed=0</code> at the end of the log then all tests passed:
| |
| | |
| localhost: ok=29 changed=11 unreachable=0 failed=0
| |
| | |
| For more detailed test results check the <code>test.log</code> and other files in the <code>artifacts</code> directory:
| |
| | |
| vim artifacts/test.log
| |
| | |
| That's it! You just executed test coverage for the Bash package :)
| |
| | |
| = Test Subjects =
| |
| | |
| To execute tests against different test subjects we need to prepare the environment. Let's store the detailed test results in <code>/tmp/artifacts</code>, use dynamic inventory as defined by the [[CI/Standard_Test_Roles|Standard Test Roles]] and download the latest Atomic Host image.
| |
| | |
| export TEST_ARTIFACTS=/tmp/artifacts
| |
| export ANSIBLE_INVENTORY=/usr/share/ansible/inventory
| |
| curl -Lo /tmp/atomic.qcow2 https://getfedora.org/atomic_qcow2_latest
| |
| | |
| Now let's try to run tests against all supported test subjects.
| |
| | |
| == Classic ==
| |
| | |
| Run tests against classic rpms installed on the system:
| |
| | |
| export TEST_SUBJECTS=''
| |
| ansible-playbook --tags=classic tests.yml
| |
| | |
| See [[CI/Standard_Test_Roles#Classic|Classic]] for detailed docs.
| |
| | |
| == Container ==
| |
| | |
| Run tests in a docker container:
| |
| | |
| export TEST_SUBJECTS=docker:docker.io/library/fedora:latest
| |
| ansible-playbook --tags=container tests.yml
| |
| | |
| See [[CI/Standard_Test_Roles#Container|Container]] for detailed docs.
| |
| | |
| == Atomic ==
| |
| | |
| Run tests against the Atomic Host:
| |
| | |
| export TEST_SUBJECTS=/tmp/atomic.qcow2
| |
| ansible-playbook --tags=atomic tests.yml
| |
| | |
| See [[CI/Standard_Test_Roles#Atomic|Atomic]] for detailed docs.
| |
| | |
| = Hints =
| |
| | |
| == Debug ==
| |
| | |
| Would you like to investigate why a test failed? Enable debugging to easilly connect to running Atomic or Container to investigate:
| |
| | |
| export TEST_DEBUG=1
| |
| ansible-playbook --tags=atomic tests.yml
| |
| | |
| See [[CI/Standard_Test_Roles#Debug|Debug]] for details about debugging.
| |
| | |
| == Ignore ==
| |
| | |
| Use <code>.gitignore</code> to specify files that Git should ignore. Such files are created during tests run. Create a <code>tests/.gitignore</code> file with the following contents:
| |
| | |
| # Ignore tests runs/artefacts.
| |
| artifacts/**
| |
| **/*.retry
| |
| | |
| = Contributing =
| |
| | |
| Are you interested in contributing a new test coverage? You are most welcome! As you have seen [[CI/Tests#Executing|Executing]] a test is quite easy. [[CI/Tests#Writing|Writing]] a new test or [[CI/Tests#Wrapping|Wrapping]] an existing one is quite simple as well. Here's a few recommendations for creating a new pull request.
| |
| | |
| Unless you are maintainer of the package, who has direct commit access, create a fork of the package git repository using the Fork button in [https://src.fedoraproject.org/rpms/bash Pagure] web interface and add your private fork as a new remote. Create a branch for your new tests. For example:
| |
| | |
| git remote add fork ssh://psss@pkgs.fedoraproject.org/forks/psss/rpms/bash.git
| |
| git checkout -b tests
| |
| | |
| Create new test coverage under the <code>tests</code> directory, update the <code>tests.yml</code> file accorgingly or create a new one. Run tests and verify they are stable and working fine in all supported environments. Add files to git, commit and push:
| |
| | |
| git add tests.yml test1 test2 test3
| |
| git commit -m "Add CI tests using the Standard Test Interface"
| |
| git push fork tests:tests
| |
| | |
| It is a good idea to include more details and links in the commit message to make the pull request easier for review:
| |
| | |
| Add CI tests using the Standard Test Interface
| |
|
| |
| Adding initial set of basic functionality tests for bash
| |
| according to the Standard Test Interface [1]. See Quick Start
| |
| Guide for brief introduction about how to run these tests [2].
| |
|
| |
| [1] https://fedoraproject.org/wiki/CI/Standard_Test_Interface
| |
| [2] https://fedoraproject.org/wiki/CI/Quick_Start_Guide
| |
| | |
| Create a new pull request from your <code>tests</code> branch against the master branch in the [https://src.fedoraproject.org/fork/psss/rpms/bash Pagure] web interface. You might want to include an additional info about the tests such as:
| |
| | |
| There are three tests available: smoke and func have been tested
| |
| across all environments (classic, container, atomic), login is
| |
| relevant for classic only (because of a missing dependency).
| |
| Please, merge the tests into all currently supported branches.
| |
| | |
| {{admon/important|Pull Requests Workaround|If you are not a member of the Fedora packager group see the [[CI/Pull_Requests|Pull Requests]] page for a temporary workaround to create a pull request from a remote repository.}}
| |