A proven tester, also known as a critical path wrangler, verifies and reports on the stability of test updates to critical path packages. They retrieve their updates from the updates-testing repository and report their findings as karma using Bodhi. Positive karma from a proven tester is required for each critical path update before it can be pushed to the stable repository.
Individuals must must be mentored and approved before becoming proven testers. A proven tester is a member of the proventesters group.
Proven testers verify a basic level of stability before releasing an update to the public. They do not need to test for total correctness or ensure complete test coverage. Some tests will vary depending on the type of the package, while others must pass for all updates.
Proven testers verify updates by installing them from the updates-testing repository. To ensure rapid detection of regressions, a full system update from this repository should be performed at least once a day. Individual packages may be updated more quickly if the need for verification is urgent.
- The system must be able to shut down and reboot.
- The user must be able to log in to the desktop.
- The user must be able to access the network.
If the package is an application, run the application and check that basic operations work.
If it is a library or other shared component, run an application which uses the component and ensure that that works.
If the component works as expected, post positive feedback. If you identify a major problem, post negative feedback. If you identify a problem which is minor in nature and does not impede the actual critical path functionality, please do not post negative feedback. Post neutral or positive feedback with a note of the issue encountered (and a link to a bug report if appropriate).
to list all installed packages from the updates-testing repository and allow to file feedback on each one at a time.
- Pay particular attention to updates whose description notes that they are critical path updates. If you identified any serious problems in earlier testing and were able to identify the package responsible, post negative feedback for that package. If possible, please file a bug report on the problem and link to the bug report in your feedback message. A good feedback message quickly and clearly identifies the behavior change and the cause, if you were able to determine it.
- If you are not sure what the component does or how to test it, do not post positive or negative feedback. If the above general tests of booting, network functionality and update functionality identified no problems, it is fine to leave a neutral feedback message noting that you were able to boot the system and perform critical path tasks with the update installed.
- If your testing uncovers no problems but you see that another tester has identified a serious problem with the package, please try to replicate their problem, and post negative feedback if you are now able to confirm it. If you are not able to confirm the problem but you suspect this may be because you cannot recreate the necessary conditions, please post neutral feedback noting that you were unable to duplicate the problem. Only post positive feedback if you are sure your testing indicates the other reporter's negative feedback is a mistake.
Since a proven tester's karma determines whether an update is allowed to be promoted, they follow special procedures based on the severity of regressions they encounter. A bug or regression is considered severe if it violates the Fedora Release Criteria or breaks a critical path action.
- Major bug - Report, vote down.
- Minor bug - Report, vote up/neutral
- Previously reported bug - Confirm, vote accordingly
- Unreproducible bug
- Unfamiliar package