From Fedora Project Wiki

(Listing of update proposals)
 
(Initial stab at including kparal's draft)
Line 163: Line 163:


=== Critique ===
=== Critique ===
== kparal's proposal ==
{{User:Kparal/Proposal:Package_update_policy}}
=== Critique ===


[[Category:Policy]]
[[Category:Policy]]
[[Category:Draft documentation]]
[[Category:Draft documentation]]

Revision as of 20:07, 9 March 2010

Warning.png
This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

There are currently several competing proposals for what an update policy should look like. Listing them all on this page.

mjg's proposal

Introduction

We assume the following axioms:

1) Updates to stable that result in any reduction of functionality to the user are unacceptable.

2) It is impossible to ensure that functionality will not be reduced without sufficient testing.

3) Sufficient testing of software inherently requires manual intervention by more than one individual.

Proposal

The ability for maintainers to flag an update directly into the updates repository will be disabled. Before being added to updates, the package must receive a net karma of +3 in Bodhi.

It should be noted that this does not require that packages pass through updates-testing. The package will appear in Bodhi as soon as the update is available. If sufficient karma is added before a push occurs, and the update is flagged for automatic pushing when the karma threshold is reached, the update will be pushed directly to updates.

It is the expectation of Fesco that the majority of updates should easily be able to garner the necessary karma in a minimal space of time. Updates in response to functional regressions should be coordinated with those who have observed these regressions in order to confirm that the update fixes them correctly.

At present, this policy will not apply to updates that are flagged as security updates.

The future

Defining the purpose of Fedora updates is outside the scope of Fesco. However, we note that updates intended to add new functionality are more likely to result in user-visible regressions, and updates that alter ABI or API are likely to break local customisations even if all Fedora packages are updated to match. We encourage the development of mechanisms that ensure that users who wish to obtain the latest version of software remain able to do so, while not tending to introduce functional, UI or interface bugs for users who wish to be able to maintain a stable platform.

Critique

mether's proposal

Requirements for Critpath packages

  • Must go through updates-testing repository
  • Only major bug fixes and security fixes.
  • Must go through updates testing repository even for security fixes

Rationale: Expedited security fixes have caused some serious regressions in the past (D-Bus, Bind, Thunderbird updates etc).

  • Requires QA team to sign off on these updates and I will leave them

to define the criteria for it. I believe the criteria should be based on feedback from testers rather than the number of days.

  • Exceptions or expedited update requests must go via release engineering


Critical_Path_Packages critical path packages list

Requirements for Non-critical path packages

  • Don't blindly push every upstream release as update
  • Preserve stability and avoid unexpected changes and push updates with

enhancements only if the benefit is considered worth the risk of potential regressions

Recommendations

  • Run AutoQA on all updates
  • Hookup PackageKit to updates-testing repo and allow users to opt-in

and provide feedback easily

  • Evaluate extending the criteria based on how well we succeed with a

more conservative update policy for critical path packages

Critique

notting's Proposal

Introduction

We assume the following axioms:

1) Updates to stable that result in any reduction of functionality to the user are unacceptable.

2) It is impossible to ensure that functionality will not be reduced without sufficient testing.

3) Sufficient testing of software inherently requires manual intervention by more than one individual.


Proposal

For a package to be pushed to the stable updates repository, it must meet the following criteria.

  1. All updates (even security) must pass AutoQA tests. Rationale: If a package breaks dependencies, does not install, or fails other obvious tests, it should not be pushed. Period. Obviously, this proposal would not be enacted until AutoQA is live.
  2. Updates that constitute a part of the 'important' package set (defined below) must follow the rules as defined for critical path packages for pending releases, meaning that they require positive karma from releng and/or QA before they go stable. This also includes security updates for these packages. The 'important' package set is defined as the following:
    • The current critical path package set
    • All major desktop environments' core functionality (GNOME, KDE, XFCE, LXDE)
    • Package updating frameworks (gnome-packagekit, kpackagekit)
    • Major desktop productivity apps. An initial list would be firefox, kdebase (konqueror), thunderbird, evolution, kdepim (kmail).

    Rationale: These are the sets of packages where regressions most affect users, and would most prevent them from Getting Their Work Done. Furthermore, while I can accept that there may be some packages in Fedora that cannot find a significant enough testing base for all potential updates, I reject the notion that any desktop widely used enough that we deploy a image or spin for it would fit into that category. I accept that this places a larger burden on QA, and would expect them to be able to contribute testing to this initiative.

  3. All other non-security updates must either:
    1. reach their specified bodhi karma threshold
    2. spend some minimum amount of time in updates-testing, with a tracked number of downloads.

    Proposed time would be one week, but is open to negotiation.

    Rationale: We do want additional eyes on updates wherever possible. We do have one Fedora mirror that Fedora infrastructure controls; we should be able to mine this server for data on updates-testing downloads.

Any update that wants to bypass these procedures would need majority approval from FESCo.

Critique

kparal's proposal

Warning.png
This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

Introduction

The purpose of this document is to present a policy/workflow that will be used when handling package updates. It specifies when, where and which kind of package updates may occur.

Scope

  • This policy concerns only currently supported Fedora releases.
    • Rules for the current pending release will be added to this document. Rules for Rawhide may come in the future.
  • Security updates that fall into the Important package set are subject to this policy. Security updates that do not fall into this category are not subject to this policy.

Workflows

Two workflows are defined, for different sets of packages. The sets are important packages and other packages.

Important package set

The important package set consists of:

  • The current critical path package set
  • All major desktop environments' core functionality (GNOME, KDE, XFCE, LXDE)
  • Package updating frameworks (gnome-packagekit, kpackagekit)
  • Major desktop productivity apps. An initial list would be firefox, kdebase (konqueror), thunderbird, evolution, kdepim (kmail).

Other package set

The other package set consists of all packages not in the important package set.

Important workflow

All package updates are submitted to the 'updates-testing' repository. Then they await initial sanity test acceptance, a process owned by the Quality Assurance team. Acceptance is granted according to Initial requirements. The decision is based on the result of Package update acceptance test plan, which is executed automatically by AutoQA. When the approval is granted, the update is pushed to the 'updates-testing' repository.

Next, the updates must meet all of these requirements:

  • Updates must reach a specified positive Bodhi karma threshold [1] provided by a defined team (e.g. proventesters). These qualified testers should consider feedback (especially negative feedback) from other testers in deciding whether to provide approval, as well as their own tests.
  • Updates must spend some minimum amount of time [1] in updates-testing. This requirement does not hold for security updates.

After submitting to 'updates' repository another round of automated AutoQA acceptance tests is run. Acceptance is granted according to Stable requirements. As a final point Release Engineering ensures that Stable requirements were satisfied and they sign off and push the update to 'updates'.


Package update policy important workflow.png

Other workflow

All package updates are submitted to the 'updates-testing' repository. Then they await initial sanity test acceptance, a process owned by the Quality Assurance team. Acceptance is granted according to Initial requirements. The decision is based on the result of Package update acceptance test plan, which is executed automatically by AutoQA. When the approval is granted, the update is pushed to the 'updates-testing' repository.

Next, the updates must meet at least one of these requirements:

  • Updates must reach a specified positive Bodhi karma threshold [1]. Anyone may provide karma to an update.
  • Updates must spend some minimum amount of time [1] in updates-testing.

Proposed time would be one week, but is open to negotiation. We can track downloads with our one Fedora-infrastructure controlled mirror as mechanism to see what usage the package is getting.

When at least one of the constraints is satisfied the package maintainer may submit the update to the 'updates' repository.

After submitting to 'updates' repository another round of automated AutoQA acceptance tests is run. Acceptance is granted according to Stable requirements. As a final point Release Engineering ensures that Stable requirements were satisfied and they sign off and push the update to 'updates'.


Package update policy other workflow.png

Initial Requirements

  • AutoQA: Test Package Sanity - Packages must be able to be installed, removed, upgraded, etc.
  • AutoQA: Packages must not break dependencies in updates + updates-testing.
  • AutoQA: Packages must not introduce file/package conflicts in updates + updates-testing.
  • + maybe some more

Stable Requirements

  • AutoQA: Initial requirements are re-checked and met.
  • AutoQA: Packages must not break upgrade path.
  • RelEng: The package update type falls into the list of allowed update types (document not approved yet, just a sample).
  • + maybe some more


Exception Process

Package maintainers may ask for exception, when:

  • they don't agree with QA rejecting their update
  • they need to push an update without fully adhering to policy requirements
    • this is suitable for high-important bug-fix updates, which repair severe regressions/breakages
    • some other candidates?
  • they don't agree with RelEng rejecting their update

The process of asking for an exception is yet to be defined. It can be a majority vote from FESCo.

Responsibilities

  • When a serious problem is found in a not yet released updated package, package update builder is responsible for removing this update from 'updates-testing' repository. Fedora Update System must provide this option for package update builder, package maintainer, Quality Assurance team and Release Engineering team.
  • The Fedora Update System is responsible for sending notifications to package update builder and package maintainer when:
    • the update receives a result from Package update acceptance test plan
    • the update has fulfilled requirements for submitting into 'updates' repository
    • the update has been accepted or rejected to be pushed to 'updates' repository by Release Engineering team
    • the update becomes available in the 'updates' repository
  • Even though some security updates do not fall within this policy, the Quality Assurance team is still obliged to test the packages, after they're pushed if necessary, and report any regressions that need to be fixed to the maintainer.

References

  1. 1.0 1.1 1.2 1.3 value to be defined

Related links

Critique