From Fedora Project Wiki

m
(some more content. moar! content!)
Line 24: Line 24:
 
* The [[Template:Validation_results|template system]] for generating result pages, the [[Template:CurrentFedoraCompose|CurrentFedoraCompose template]], the wikitcms module, and the relval utility for generating result pages and reporting results were all introduced during the Fedora 21 cycle
 
* The [[Template:Validation_results|template system]] for generating result pages, the [[Template:CurrentFedoraCompose|CurrentFedoraCompose template]], the wikitcms module, and the relval utility for generating result pages and reporting results were all introduced during the Fedora 21 cycle
 
* Individual nightly compose result pages and [https://www.happyassassin.net/2014/12/24/fedora-22-and-later-nightly-testing/ automatic generation of nightly result pages] were introduced in [[:Category:Fedora_22_Nightly_Test_Results|Fedora 22]]
 
* Individual nightly compose result pages and [https://www.happyassassin.net/2014/12/24/fedora-22-and-later-nightly-testing/ automatic generation of nightly result pages] were introduced in [[:Category:Fedora_22_Nightly_Test_Results|Fedora 22]]
 +
 +
== Features ==
 +
 +
The following can be considered as features of the wikitcms 'system':
 +
 +
* Storage and organization (as pages, in categories) of [[:Category:Test_Cases|test cases]]
 +
* Advanced formatting and templating for test case content (via the mediawiki engine)
 +
* Result reporting without authentication
 +
* Capability to distinguish between test case and 'test instance'
 +
* Results sorted by environment (with arbitrary 'environments')
 +
* Tests for a given build grouped by 'test type' and section within test type
 +
 +
== Implementation ==
 +
 +
Fundamentally, wikitcms consists of a set of conventions and expectations about the ''names'', ''contents'' and ''categorization'' of wiki pages related to test results. There are also several critical 'auxiliary' pages which are used in the generation of the primary pages; these are considered a part of the system and should be modified only with care.
 +
 +
=== Names ===
 +
 +
* Release validation result pages are in the ''Test Results'' namespace, and are named ''Fedora '''RELEASE''' '''MILESTONE''' '''COMPOSE''' '''TESTTYPE''''' or ''Fedora '''RELEASE''' '''MILESTONE''' '''DATE''' '''TESTTYPE''', where:
 +
*: '''RELEASE''' is a Fedora release version number
 +
*: '''MILESTONE''' is a valid Fedora milestone for the release, or a nightly compose type, currently ''Rawhide'' or ''Branched''
 +
*: '''COMPOSE''' is a TC or RC compose identifier, e.g. ''TC1'' or ''RC2'' (nb: ''RC2.1'' is a valid format here)
 +
*: '''DATE''' is a nightly compose date, in ''YYYYMMDD'' format
 +
*: '''TESTTYPE''' is one of the validation test types (see below)
 +
* Validation test category pages are named as follows:
 +
*: The top-level [[:Category:Test_Results]] exists
 +
*: Each Fedora release has a ''Fedora '''RELEASE''' Test Results'' category that is a member of [[:Category:Test_Results]]
 +
*: There is a category for each milestone for each Fedora release, which is a member of that release's category, with the name ''Fedora '''RELEASE''' '''MILESTONE''' Test Results''
 +
*: If there are nightly validation pages for the release, there is a category with the name ''Fedora '''RELEASE''' Nightly Test Results'', which is a member of the that release's category

Revision as of 01:38, 21 January 2015

Wikitcms is a term used to refer to Fedora's use of this Mediawiki instance as an ad hoc test management system ('TCMS' stands for 'Test Case Management System', but that precise initialism is in fact quite rarely used). It is also the name of a Python module which provides an interface to the 'system'.

Background

The use of the Fedora wiki to track test results dates back to at least the page QA/FC6Test2TreeTesting - at that time, the Fedora wiki was a MoinMoin instance, not this Mediawiki instance. The rough form of a table with different tests as the rows and results as columns is visible even there. Since then, this basic format has been gradually elaborated.

Features

The following can be considered as features of the wikitcms 'system':

  • Storage and organization (as pages, in categories) of test cases
  • Advanced formatting and templating for test case content (via the mediawiki engine)
  • Result reporting without authentication
  • Capability to distinguish between test case and 'test instance'
  • Results sorted by environment (with arbitrary 'environments')
  • Tests for a given build grouped by 'test type' and section within test type

Implementation

Fundamentally, wikitcms consists of a set of conventions and expectations about the names, contents and categorization of wiki pages related to test results. There are also several critical 'auxiliary' pages which are used in the generation of the primary pages; these are considered a part of the system and should be modified only with care.

Names

  • Release validation result pages are in the Test Results namespace, and are named Fedora RELEASE MILESTONE COMPOSE TESTTYPE or Fedora RELEASE MILESTONE DATE TESTTYPE, where:
    RELEASE is a Fedora release version number
    MILESTONE is a valid Fedora milestone for the release, or a nightly compose type, currently Rawhide or Branched
    COMPOSE is a TC or RC compose identifier, e.g. TC1 or RC2 (nb: RC2.1 is a valid format here)
    DATE is a nightly compose date, in YYYYMMDD format
    TESTTYPE is one of the validation test types (see below)
  • Validation test category pages are named as follows:
    The top-level Category:Test_Results exists
    Each Fedora release has a Fedora RELEASE Test Results category that is a member of Category:Test_Results
    There is a category for each milestone for each Fedora release, which is a member of that release's category, with the name Fedora RELEASE MILESTONE Test Results
    If there are nightly validation pages for the release, there is a category with the name Fedora RELEASE Nightly Test Results, which is a member of the that release's category