From Thursday January 18th 2018, bodhi has been adjusted to gate updates based on test results.
Here some information about this process.
What gates?
Currently there are three tests allowed to gate an update:
- dist.rpmdeplint - to make sure the update's dependencies are available
- dist.abicheck - to make sure the update's ABI remains stable in stable Fedora releases
- AtomicCI pipeline - for packages that are part of the Atomic Host and include in their dist-git repository tests according to the specifications.
This last requirement only concern packages that are in Fedora Atomic Host while the first two are enforced for all packages.
Note: At the moment, Bodhi only queries Greenwave every 6 hours. If you see a message like "1 of 2 required tests not found", with the results already shown in the "automated tests" tab, the situation will get rectified with the next Bodhi-Greenwave sync.
Package level testing
We are currently working on building a pipeline that will support package level testing for all packages interested in adding tests to their dist-git repositories.
How to fix the tests?
The tests are all in your hand, you can fix the dist.depcheck and dist.abicheck by adjusting the update or the build and you can fix the package level testing since the tests are stored in the dist-git repository of the package itself. You have the control on the tests.
Go through
There are cases where one will want to go through the gating despite the tests failing. This is currently allowed using the waiverdb-cli command line tool (dnf install waiverdb-cli). See more at https://fedoraproject.org/wiki/Package_update_HOWTO#Handling_feedback_from_automated_tests
We are working on integrating this into bodhi itself to make waiving test results easier. See more at https://github.com/fedora-infra/bodhi/pull/2095