This SOP describes the best practices for creating test cases for Fedora.
Background
A test case is a set procedure with a limited range of possible results, intended to be followed in order to test something. Test cases can be used to test all sorts of things, not just software code. Test cases for anything in Fedora can be created by anyone - you do not need to be involved in the creation or maintenance of a particular component or process to create a test case for it.
Principles
In creating test cases, please keep the following principles in mind.
Test one thing at a time
A test case should be single-purpose: each test case should test one function. Do not write one giant test case to test twenty different things. If you need to test multiple functions, write several test cases and join them together with a test plan.
Be comprehensive
Try to make the description of the test process and the possible results as comprehensive and detailed as possible. Try to think of any assumptions you may be making about the process and instead write them out explicitly. Examples include specifying whether a test should start with a clean installation of the software to be tested (or the distribution as a whole), or whether to create some initial configuration before running a test. Try to place yourself in the shoes of someone who is entirely unfamiliar with the function to be tested, and make sure they can successfully complete the test simply by following the procedure. Feel free to link to more explicit instructions elsewhere in the Wiki or on the web for any complex steps. Try to make your instructions as accessible as practical to those without advanced knowledge of the topic area, and use simple English.
Be generic
As far as is possible without compromising comprehensiveness, try to write the test case such that it will not go 'stale'. Try to ensure your test case will be usable for a long time without regular editing. This principle is best illustrated by examples: for instance, in writing a test to determine whether audio works on a default Fedora installation, you should not assume that the default desktop will always be what it currently is, or that the same set of applications will always be installed, so you should not write instructions that are specific to a particular desktop's method for checking volume settings, or to a particular media player. This also means not to include instructions in a test case which are specific to a particular event for which you are initially creating the test case: write the test case so it will be generically useful long past any specific test event, and include additional, event-specific test instructions in the page for the event. More generic tests can more easily be adapted to apply to cases you may not have considered, and will not become rapidly useless if you are not in a position to maintain them.
Naming
We have chosen not to prescribe a particular format for naming test case pages (aside from always starting them with QA:Testcase_ , as described below). We would recommend that you follow a consistent scheme in naming related test cases, and look at test cases related to or similar to any test case you are about to create and try to follow any conventions they set.
Process
To actually create a test case, use the test case template. If you visit that page, you will see instructions for how to use the template in another page. Create a page for your test case. The names of test case pages should always begin QA:Testcase_. Use the template and fill in the description section with a brief description of the test case, the setup section with pre-test setup instructions if appropriate, the actions section with the procedure for running the test, and the results section with the expected outcome of the test (but do not include non-generic instructions for what to do in the case of other outcomes: see #Be generic). Add the page to the appropriate categories; these will depend on the type of test case. You should usually not add it to the Category:Test_Cases category directly, unless it cannot sensibly go into any more specific category. If the test case you wish to create relates to a specific Fedora package, see the package test plan creation SOP for instructions on categorization.
Test scripts
Some test cases might involve running multiple commands with several different options, and inspecting the output. It might be helpful to write a test script so other testers can run the desired commands, or to inspect the output or log files, in a consistent manner. Do not include any test script longer than 5 lines of code in the test case template. Instead, link to the test script, or upload it to the wiki and add the script to the Category:Test_Scripts category.