From Fedora Project Wiki

(โ†’โ€ŽUser switching: add a test case reference, see https://pagure.io/fedora-qa/issue/630)
(fix {{anchor|post-install-requirements}} that got moved accidentally, rename the new anchors to url-friendly format)
 
(2 intermediate revisions by 2 users not shown)
Line 185: Line 185:


{{anchor|windows-dual-boot}}
{{anchor|windows-dual-boot}}
==== Windows dual boot ====
==== Windows dual boot ====
The installer must be able to install into free space alongside an existing clean Windows installation and install a bootloader which can boot into both Windows and Fedora.
The installer must be able to install into free space alongside an existing clean Windows installation and install a bootloader which can boot into both Windows and Fedora.
Line 254: Line 253:
{{anchor|post-install-requirements}}
{{anchor|post-install-requirements}}
=== <span style="text-decoration:underline">Post-install requirements</span> ===
=== <span style="text-decoration:underline">Post-install requirements</span> ===
{{anchor|first-boot-experience}}
==== First boot experience ====
If an initial setup utility is run or intended to be run after the first boot of the installed system, then it must start successfully and each page or panel of the initial setup utility should withstand a basic functionality test.
{{hidden|header=References|content=
* Proposed [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/HSUETWLZIXFKJXGQUEJ2QAOQZCDG242B/ 2020-09-01], approved [https://meetbot.fedoraproject.org/fedora-meeting/2020-09-14/fedora-qa.2020-09-14-15.01.html 2020-09-14], implemented 2020-09-15
* Test case: [[QA:Testcase base initial setup]]
|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
{{anchor|system-services}}
{{anchor|system-services}}
==== System services ====
==== System services ====
Line 354: Line 362:


{{anchor|default-panel-functionality}}
{{anchor|default-panel-functionality}}
==== Default panel functionality ====
==== Default panel functionality ====
All elements of the default panel (or equivalent) configuration in all release-blocking desktops must function correctly in typical use.
All elements of the default panel (or equivalent) configuration in all release-blocking desktops must function correctly in typical use.
Line 485: Line 492:


{{anchor|domain-sso}}
{{anchor|domain-sso}}
==== Single sign-on ====
==== Single sign-on ====
* It must be possible for users to perform passwordless single-sign-on between two properly-configured domain clients using GSSAPI.
* It must be possible for users to perform passwordless single-sign-on between two properly-configured domain clients using GSSAPI.
Line 511: Line 517:
|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}


{{anchor|server-product-requirements}}
{{anchor|server-edition-requirements}}
=== <span style="text-decoration:underline">[[Server]] Product requirements</span> ===
=== <span style="text-decoration:underline">[[Server]] Edition requirements</span> ===
These requirements apply only to the Server product.
These requirements apply only to the Server edition.


{{anchor|role-functional-requirements}}
{{anchor|role-functional-requirements}}
Line 536: Line 542:
{{hidden|header=References|content=
{{hidden|header=References|content=
* Proposed [https://lists.fedoraproject.org/pipermail/server/2015-November/002114.html 2015-11-05], implemented [http://meetbot.fedoraproject.org/fedora-meeting-1/2015-11-10/serversig.2015-11-10-16.02.html 2015-11-10]
* Proposed [https://lists.fedoraproject.org/pipermail/server/2015-November/002114.html 2015-11-05], implemented [http://meetbot.fedoraproject.org/fedora-meeting-1/2015-11-10/serversig.2015-11-10-16.02.html 2015-11-10]
|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
{{anchor|IoT-Edition-requirements}}
=== <span style="text-decoration:underline">IoT Edition requirements</span> ===
{{anchor|greenboot-service}}
==== Greenboot service ====
The Greenboot service must be present on all images and installed by default when using the ISO installer. The service must be enabled to run on boot and function as intended.
{{hidden|header=References|content=
* Test cases: [[QA:Testcase_Greenboot]] as listed on [[Template:General_IoT_test_matrix]]
|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
{{anchor|automatic-partition-decryption}}
==== Automatic partition decryption (''Clevis'') ====
When configured on hardware with a Trusted Platform Module (TPM), it must be possible for the Clevis utility to automatically decrypt any encrypted partitions on boot without any user intervention.
{{hidden|header=References|content=
* Test cases: [[QA:Testcase_Clevis]] as listed on [[Template:General_IoT_test_matrix]]
|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}
|headerstyle=background:#e5e5e5|fw1=normal|ta1=left}}



Latest revision as of 11:03, 7 October 2020

๐Ÿ”— Final Objectives

The objective of the Final release is to:

  1. Provide a polished final release suitable for meeting the needs of our Target Audience

๐Ÿ”— Final Release Requirements

In order to be released to the general public, a compose must meet all of the following criteria. This is intended to make the decision process as clear and straightforward as possible. Mostly met items are incomplete until they are met. Optional and nice to have items should not be included in this list.

There may be times where a requirement is unmet only in a particular configuration, such as with some keyboard layouts but not others, or if a particular character is used in a username, password or passphrase. In such cases, the release team should use their judgement and refer to precedent to determine whether or not the issue should be considered to block the release. They should consider the number of users likely to be affected by the issue, the severity of the case when the issue is encountered, and the ease or otherwise with which the issue can be avoided by both informed and uninformed users.

The term release-blocking desktops means all the desktop environments in which bugs are currently considered capable of blocking a Fedora release. The current set of release-blocking desktops for x86_64 is GNOME and KDE, and for aarch64 is GNOME. Note that bugs in desktops that are not part of this set which would infringe these criteria automatically qualify for freeze exception status, according to the freeze exception bug process.

The term release-blocking images means all the images in which bugs are currently considered capable of blocking a Fedora release. The current set of release-blocking images includes the images defined by the three Editions - Server, Workstation and IoT - in their product requirement documents and/or technical specifications, the Everything network install image, key Cloud images, and the KDE live image. The canonical list of release-blocking images for Fedora 42 is on this page.

๐Ÿ”— Process requirements

๐Ÿ”— Beta criteria met

All Fedora 33 Beta Release Criteria must be met.

๐Ÿ”— Final blockers CLOSED

All bugs blocking the Final tracker must be CLOSED.

๐Ÿ”— Edition requirements

This section contains requirements relating to Fedora Editions: not to any specific edition, but to editions in general.

๐Ÿ”— Self-identification

For each Edition, the installer must be clearly identified as that Edition in all of that Edition's installable release-blocking images. Also, for any deployment of that Edition in accordance with the rest of these criteria, the relevant fields in /etc/os-release must include the correct Edition name. Release-blocking images that are not part of any edition must not identify themselves as being part of any edition.

References

๐Ÿ”— Initialization requirements

๐Ÿ”— Release-blocking images must boot

All release-blocking images must boot in their supported configurations.

Supported media types
Difference from Beta
QA Testing responsibilities

๐Ÿ”— 'Basic graphics mode' boot mode behavior

The generic video driver option ('basic graphics mode' - as described in the Basic criteria) on all release-blocking installer and live images must function as intended (launching the installer or desktop and attempting to use a generic driver), and there must be no bugs that clearly prevent the installer or desktop from being reached in this configuration on all systems or on wide classes of hardware.

References

๐Ÿ”— No broken packages

There must be no errors in any package on the release-blocking images which cause the package to fail to install.

What errors?
Automatic blockers
References

๐Ÿ”— Installer requirements

๐Ÿ”— Media consistency verification

Validation of install media must work correctly for all release-blocking images.

Details
References

๐Ÿ”— Live image persistent overlays

The release-blocking live images must properly support mounting and using a persistent storage overlay for the entire system and/or one for the /home partition.

Writing not covered
References

๐Ÿ”— Package and installer sources

The installer must be able to use all supported local and remote package and installer sources.

๐Ÿ”— Network attached storage

The installer must be able to detect (if possible) and install to supported network-attached storage devices.

Supported types
References

๐Ÿ”— Installation interfaces

The installer must be able to complete an installation using all supported interfaces.

References

๐Ÿ”— Minimal install

When installing with the generic network install image with no update repositories enabled, the installer must be able to install the minimal package set.

References

๐Ÿ”— Disk layouts

The installer must be able to create and install to any workable partition layout using any file system and/or container format combination offered in a default installer configuration.

Wait, what?
Default installer configuration?
References

๐Ÿ”— Firmware RAID

  • The installer must be able to detect and install to firmware RAID storage devices.
System-specific bugs
References

๐Ÿ”— Bootloader disk selection

The installer must allow the user to choose which disk the system bootloader will be installed to, and to choose not to install one at all.

References

๐Ÿ”— Storage volume resize

Any installer mechanism for resizing storage volumes must correctly attempt the requested operation.

Unknown volumes
What does that cover?
Corruption-causing bugs covered in another criterion

๐Ÿ”— Windows dual boot

The installer must be able to install into free space alongside an existing clean Windows installation and install a bootloader which can boot into both Windows and Fedora.

Clean Windows installation?
Secure Boot excepted
References

๐Ÿ”— OS X dual boot

The installer must be able to install into free space alongside an existing OS X installation, install and configure a bootloader that will boot Fedora.

References

๐Ÿ”— Update image

The installer must be able to use an installer update image retrieved from removable media or a remote package source.

Supplementary to Basic

๐Ÿ”— Installer translations

The installer must correctly display all sufficiently complete translations available for use.

Substantial breakage only

๐Ÿ”— Installer help

Any element in the installer interface(s) which is clearly intended to display 'help' text must do so correctly when activated.

๐Ÿ”— Cloud image requirements

๐Ÿ”— Cloud images work in EC2

Release-blocking cloud disk images must be published to Amazon EC2 as AMIs, and these must boot successfully and meet other relevant release criteria on at least one KVM-based x86 instance type, at least one KVM-based aarch64 instance type, and at least one Xen-based x86 instance type.

๐Ÿ”— Post-install requirements

๐Ÿ”— First boot experience

If an initial setup utility is run or intended to be run after the first boot of the installed system, then it must start successfully and each page or panel of the initial setup utility should withstand a basic functionality test.

References

๐Ÿ”— System services

All system services present after installation with one of the release-blocking package sets must start properly, unless they require hardware which is not present.

References

๐Ÿ”— Keyboard layout configuration

If a particular keyboard layout has been configured for the system, that keyboard layout must be used:

  • When unlocking encrypted storage volumes during boot (but see footnotes)
  • When logging in at a console
  • When logging in via the default login manager for a release-blocking desktop
  • After logging in to a release-blocking desktop, if the user account does not have its own keyboard layout configuration for that desktop (if there is such a user/desktop-specific configuration, it must be used when that user logs in to that desktop)
Switched layouts when unlocked encrypted storage

๐Ÿ”— Critical path translations

All critical path actions on release-blocking desktops must correctly display all sufficiently complete translations available for use.

What does this cover, exactly?
References

๐Ÿ”— SELinux and crash notifications

There must be no SELinux denial notifications or crash notifications on boot of or during installation from a release-blocking live image, or at first login after a default install of a release-blocking desktop.

Exclusions
References

๐Ÿ”— Data corruption

All known bugs that can cause corruption of user data must be fixed or documented at Common F33 bugs.

User data?
Fixed or documented?
References

๐Ÿ”— Default application functionality

All applications that can be launched using the standard graphical mechanism after a default installation of Fedora Workstation on the x86_64 architecture must start successfully and withstand a basic functionality test.

For other release-blocking desktops (on any architecture), the requirements only apply to the following types of applications:

  • web browser
  • file manager
  • package manager
  • image viewer
  • document viewer
  • text editor
  • archive manager
  • terminal emulator
  • problem reporter
  • help viewer
  • system settings

If there are multiple applications of the same type (e.g. several web browsers), the primary/default one must satisfy the requirements. If the primary/default application can't be determined, at least one of said applications must satisfy the requirements.

All applications installed by default in Fedora Workstation must comply with each MUST and MUST NOT guideline in the Applications and Launchers policy.

Basic functionality
Application types examples
Determining primary applications
References

๐Ÿ”— Default panel functionality

All elements of the default panel (or equivalent) configuration in all release-blocking desktops must function correctly in typical use.

Typical use?
References

๐Ÿ”— Update notification

Release-blocking desktops must notify the user of available updates, but must not do so when running as a live image.

๐Ÿ”— Desktop keyring

Saving passwords to and retrieving passwords from the default keyring must work for all release-blocking desktops.

References

๐Ÿ”— Printing

Printing must work in release-blocking desktops on at least one printer using each of the following drivers:

  • The built-in print-to-PDF driver
  • The generic IPP driver
'Work'?
References

๐Ÿ”— Artwork

The proposed final Fedora artwork must be included and used as the background on release-blocking desktops. All Fedora artwork visible in critical path actions on release-blocking desktops must be consistent with the proposed final theme.

๐Ÿ”— Pre-release notices

No notices or alerts about pre-release status may be shown as part of installation or critical path actions on release-blocking desktops.

Installer warning
References

๐Ÿ”— Security bugs

The release must contain no known security bugs of 'important' or higher impact according to the Red Hat severity classification scale which cannot be satisfactorily resolved by a package update (e.g. issues during installation).

Determination
References

๐Ÿ”— Kickstarts

At the time of the release, the fedora-kickstarts git repository must have a tag matching the commit used to produce the accepted release candidate compose. The tag's name must clearly and unambiguously match the release number.

References

๐Ÿ”— Release notes

Any release notes included in the release repositories and/or any release-blocking deliverables must be for the correct release, and approved for release by the documentation team.

References

๐Ÿ”— Release identification

A fedora-release package containing the correct names, information and repository configuration for a final Fedora release must be present on release-blocking images and the appropriately versioned generic-release package must be available in the release repository.

References

๐Ÿ”— User switching

User switching must work using the mechanisms offered (if any) by all release-blocking desktops in their default configuration.

What is user switching?
Work?
Default configuration?
References

๐Ÿ”— Domain client requirements

These requirements apply to any system properly configured as a client of another system which is an active and correctly configured FreeIPA domain controller.

๐Ÿ”— Access control

  • The system must honor the controller's HBAC rules for access control.

๐Ÿ”— Password changes

  • Users with domain accounts must be able to change their passwords according to the password policy specified by the domain controller.

๐Ÿ”— Single sign-on

  • It must be possible for users to perform passwordless single-sign-on between two properly-configured domain clients using GSSAPI.

๐Ÿ”— Server discovery

  • When configured to use the domain controller for DNS services, the system must be able to use DNS to discover the domain controller address using SRV records.
References

๐Ÿ”— SSH host key validation

  • When configured to use FreeIPA for host-key validation, an initial SSH connection between domain clients must not prompt the user to accept the SSH public key.
References

๐Ÿ”— Server Edition requirements

These requirements apply only to the Server edition.

๐Ÿ”— FreeIPA and PostgreSQL server requirements

All the additional Beta requirements for FreeIPA and PostgreSQL server systems must be met, without any workarounds being necessary.

References

๐Ÿ”— Cockpit management interface

All Cockpit functional criteria must be satisfied when the user is running any of the following platforms:

  • Mozilla Firefox as shipped in the same Fedora release
  • Google Chrome of the latest available version at compose time on the same Fedora release.
  • At least one of a) Mozilla Firefox or b) Google Chrome of the latest available version on Windows at compose time.
  • At least one of a) Mozilla Firefox or b) Google Chrome of the latest available version on OSX at compose time.

Manual testing will occur on the Firefox/Fedora platform. It will not be performed on the Chrome/Fedora platform nor any of the Windows / OSX platforms mentioned above, but if issues in code related to Fedora / Cockpit (and not third-party software such as the third-party browsers) in meeting the functional criteria are reported against those platforms, they may block the release. Workarounds to get around issues that are outside of our control (such as in third party browsers) will not be considered blockers but may be considered freeze exceptions.

References

๐Ÿ”— IoT Edition requirements

๐Ÿ”— Greenboot service

The Greenboot service must be present on all images and installed by default when using the ISO installer. The service must be enabled to run on boot and function as intended.

References

๐Ÿ”— Automatic partition decryption (Clevis)

When configured on hardware with a Trusted Platform Module (TPM), it must be possible for the Clevis utility to automatically decrypt any encrypted partitions on boot without any user intervention.

References


๐Ÿ”— Final Blocker Bugs

A bug is considered a Final blocker bug if any of the following criteria are met:

A Fedora Change being incomplete, in and of itself, does not constitute a blocker bug. The Change process is separate from this process. Changes are required to meet certain standards at certain points of the release cycle, but this is part of the Change process and managed, tracked and enforced separately from this process. However, if a proposed feature being incomplete causes any of the above criteria to be met, then the bug is a release blocker.


๐Ÿ”— Contingency Plan

  • If all of the Final Release Requirements are not met by 20:00 UTC on Tuesday the week prior to release day, the release will be delayed by one week so that the Final Release Requirements can be met.
  • One week will be added to all remaining tasks in the release schedule, including the final release date.
  • This decision will be made at the Go/No-Go Meeting.

๐Ÿ”— Confirming Final Requirements

QA has the responsibility of determining whether the criteria for the release has been met (as outlined above) through discussion with Development and Release Engineering. QA's findings will be reviewed and discussed at the Go/No-Go Meeting.

๐Ÿ”— Related Pages