Revision as of 17:56, 25 September 2014

Fedora Release Engineering

Introduction

The development of Fedora is a very open process, involving over a thousand package maintainers (along with testers, translators, documentation writers and so forth). These maintainers are responsible for the bulk of Fedora distribution development. An elected committee of people provides some level of direction over the engineering aspects of the project.

The rapid pace of Fedora development leaves little time for polishing the development system into a quality release. To solve this dilemma, the Fedora project makes use of regular freezes and milestone (Alpha, Beta, Final) releases of the distribution, as well as "branching" of our trees to maintain different strains of development.

Stable branches of the Fedora tree and associated Repositories are maintained for each Fedora release. The Rawhide rolling development tree is the initial entry point for all Fedora development, and the trunk from which all branches diverge. Releases are Branched from Rawhide some time before they are sent out as stable releases, and the milestone releases (Alpha, Beta and Final) are all built from this Branched tree.

Nightly snapshot images of various kinds are built from Rawhide and Branched (when it exists) and made available for download from within the trees on the mirrors or from the Koji build system. Many of these can be located via the release engineering dashboard.

The Fedora Release Life Cycle page is a good entry point for more details on these processes. Some other useful references regarding the Fedora release process include:

The Release planning process
The release validation test plan
The updates-testing process, via Bodhi and the Updates Policy
The test compose and release candidate system
The QA:SOP_blocker_bug_process and QA:SOP_freeze_exception_bug_process
The Repositories
The Bugzilla system

Final Release Checklist

Various tasks need to be accomplished prior to a final Fedora release. Release Engineering is responsible for many of them, as outlined here.

Export Approval

As a U.S. based company, Red Hat needs to file for export approval when creating Fedora releases. Currently this involves giving Red Hat Legal a heads up at the final freeze date, and updating Legal once the final tree has been spun.

Release Announcement

The Fedora Documentation Project prepares release announcements for the final releases. A bug needs to be filed for this two weeks before the final release date.

Mirror List Files

A new set of mirror list files need to be created for the new release. Email Fedora Mirror Admins to have these files created. These should be created at the Final Freeze point but may redirect to Rawhide until final bits have been staged.

Update Tool

The Fedora Update tool will need to be updated to handle update requests for the new release. Updates for the new release should start to be pushed a few days before the release date. This will allow for ensuring the update process will work, and to get eyes on the update bits before the rest of the world gets them. This will also allow those who were early adopters of Fedora to get updates after the final bits have been composed into a release.

Release Composing

Fedora “releases” can be built by anyone with a fast machine of proper arch and access to a package repository. All of the tools necessary to build a release are available from the package repository. These tools aim to provide a consistent way to build Fedora releases. A complete release can actually be built with only a couple commands, including the creation of network install images, offline install images ('DVDs'), live images, disk images, install repositories, FedUp upgrade images, and other bits. These commands are named pungi and livecd-creator.

Pungi

Pungi is the tool used to compose Fedora releases. It requires being ran in a chroot of the package set that it is composing. This ensures that the correct userland tools are used to match up with the kernel that will be used to perform the installation. It uses a comps file + yum repos to gather packages for the compose. The Koji build system provides a way to run tasks within chroots on the various arches, as well as the ability to produce yum repos of packages from specific collections.

Livecd-creator

Livecd-creator is part of the livecd-tools package in Fedora and it is used to compose the live images as part of the Fedora release. It is also used to compose many of the custom Spins or variants of Fedora.

Distribution

Once a compose has been completed, the composed tree of release media, installation trees, and frozen Repositories needs to be synchronized with the Fedora mirror system. MirrorManager has some more details on the mirror system. Many of the images are also offered via BitTorrent as an alternative method of downloading.

Download Mirrors

Depends on the Fedora Mirror System and infrastructure to populate them privately.

BitTorrent

BitTorrent is currently served by http://torrent.fedoraproject.org. Images are added to the system via this Standard Operating Procedure.

Acknowledgements

This document was influenced by release engineering documents from FreeBSD.

@@ Line 1: / Line 1: @@
-<!-- page was renamed from JesseKeating/RelEng
--->
 = Fedora Release Engineering =
-<!--only use ToC if the page will be longer than a standard browser window
--->
 {{Anchor|Introduction}}
 == Introduction ==
-The development of Fedora is a very open process. Fedora is comprised of contributions from thousands of people around the world. The Fedora Project provides anonymous [[UsingCvs|  CVS]]  access to the general public so that others can have access to log messages, diffs (patches) between development branches, and other productivity enhancements that formal source code management provides. This has been a huge help in attracting more talented developers to Fedora. However, I think everyone would agree that chaos would soon manifest if write access was opened up to everyone on the Internet. Therefore only a “select” group of nearly 2000 people are given write access to the CVS repository. These maintainers are responsible for the bulk of Fedora development. An elected [[Development/SteeringCommittee|  committee of people]]  provides some level of direction over the project.
+The development of Fedora is a very open process, involving over a thousand package maintainers (along with testers, translators, documentation writers and so forth). These maintainers are responsible for the bulk of Fedora distribution development. An elected [[Fedora_Engineering_Steering_Committee|committee of people]]  provides some level of direction over the engineering aspects of the project.
-The rapid pace of Fedora development leaves little time for polishing the development system into a  quality release. To solve this dilemma, the Fedora project makes use of regular freezes and Alpha/Beta releases of the packages that comprise Fedora, as well as "branching" of our package repository to maintain different strains of development.  The development branch is called ''devel/''.
-Stable branches are maintained for each Fedora release, named after the release (FC-6, F-7, F-8).  ''devel/'' is the “bleeding-edge” of Fedora development (commonly referred to as ''Rawhide'') where all new changes first enter the system and are fed into "Rawhide".  [[Releases/Rawhide|  Rawhide]]  is the code name given to the development tree from which major releases are made.
-In the interim periods between releases, nightly snapshot trees are composed from the rawhide collection and made available for [[Download|  download]] . The widespread availability of binary release snapshots, and the tendency of our user community to keep up with rawhide development helps to keep rawhide in a somewhat usable condition even before the quality assurance activities ramp up pending a major or pre-release.
-Bug reports and feature requests are continuously submitted by users throughout the release cycle. Problems reports are entered into our [[BugsAndFeatureRequests|  Bugzilla system]] . In addition to the multitude of different technical [[Communicate#Mailing_Lists|mailing lists]]  about Fedora, the [[QA|  Fedora Quality Assurance]]  [https://www.redhat.com/mailman/listinfo/fedora-test-list mailing list]  provides a forum for discussing the finer points of “release-polishing”.
+The rapid pace of Fedora development leaves little time for polishing the development system into a quality release. To solve this dilemma, the Fedora project makes use of regular freezes and milestone (Alpha, Beta, Final) releases of the distribution, as well as "branching" of our trees to maintain different strains of development.
-To service updates for releases, individual release branches are created before a final release is made.
+Stable branches of the Fedora tree and associated [[Repositories]] are maintained for each Fedora release. The [[Releases/Rawhide|Rawhide]] rolling development tree is the initial entry point for all Fedora development, and the trunk from which all branches diverge. Releases are [[Releases/Branched|Branched]] from Rawhide some time before they are sent out as stable releases, and the milestone releases (Alpha, Beta and Final) are all built from this Branched tree.
-{{Anchor|release_process}}
+Nightly snapshot images of various kinds are built from Rawhide and Branched (when it exists) and made available for download from within the trees on the [https://mirrors.fedoraproject.org/ mirrors] or from the [[Koji]] build system. Many of these can be located via the [https://apps.fedoraproject.org/releng-dash/ release engineering dashboard].
-== Release Process ==
+The [[Fedora Release Life Cycle]] page is a good entry point for more details on these processes. Some other useful references regarding the Fedora release process include:
-New releases of Fedora are released from the ''rawhide'' collection at approximately six month intervals.  The Fedora Development cycle is driven by the desire to have a good balance between low overhead for maintainers to get builds into a release and enough oversight to keep the tree in a stable state during release preparation.  During the development cycle, an Alpha and a Beta pre-release will be made to track the development progress.  The Fedora release process begins to ramp up roughly 21 days before the anticipated release date when the release engineer sends an email to the development mailing lists to remind developers that they only have roughly 12 days to integrate new changes before the tree freeze.
+* The [[Changes/Policy|Release planning process]]
+* The [[QA:Release_validation_test_plan|release validation test plan]]
+* The [[QA:Updates_Testing|updates-testing process]], via [[Bodhi]] and the [[Updates Policy]]
+* The [[QA:SOP_compose_request|test compose and release candidate system]]
+* The [[QA:SOP_blocker_bug_process]] and [[QA:SOP_freeze_exception_bug_process]]
+* The [[Repositories]]
+* The [[BugsAndFeatureRequests|Bugzilla system]]
-=== Feature Planning ===
+== Final Release Checklist ==
-The [[Features/Policy|  Fedora Feature Policy]]  drives how Fedora defines Features for a release.  At any time a Feature Proposal can be created.  Deadlines for getting a feature approved for a given Fedora release is the Beta Freeze for that release.  Features will be considered for approval beginning at the start of a release development cycle.
-=== Freezes and Pre-Releases ===
-{{Anchor|alpha_freeze}}
-==== Alpha Freeze ====
-Nine days before the anticipated Alpha release, the Fedora Release Engineering team makes a snapshot of ''Rawhide'' within the [[PackageMaintainers/UsingKoji|  Fedora Buildsystem]] .  This is called a ''freeze collection''.  Rawhide will continue to compose from the builds created from devel/, but QA and Release Engineering focus will be on the content of the freeze collection.  This is what we call a ''non-blocking'' freeze, as in builds entering Rawhide are not blocked by the freeze.  All builds tagged for the freeze collection must be approved by the [[ReleaseEngineering|  Release Engineering Team]] . The kinds of changes that are allowed during this short period include:
-* Bug fixes.
-* Documentation updates.
-* Security-related fixes of any kind.
-* Minor changes to device drivers, such as adding new Device IDs.
-* Any additional change that the release engineering team feels is justified, given the potential risk.
-An Alpha release will be composed from this freeze collection and mirrored for widespread testing.
-{{Anchor|beta_freeze}}
-==== Beta Freeze ====
-Nine days before the anticipated beta release, Rawhide enters a “tree slush”. During this time, a freeze collection will be created in the build system.  Rawhide will compose from this collection for the duration of the freeze, thus creating a ''blocking-freeze''.  All builds tagged for the freeze collection must be approved by the [[ReleaseEngineering|  Release Engineering Team]] . The kinds of changes that are allowed during this short period include:
-* Bug fixes.
-* Documentation updates.
-* Security-related fixes of any kind.
-* Minor changes to device drivers, such as adding new Device IDs.
-* Any additional change that the release engineering team feels is justified, given the potential risk.
-During the tree slush, a Beta release is prepared from the freeze collection (basically Rawhide) and mirrored for widespread testing.  After the Beta release is prepared and made available to mirrors, Rawhide will once again compose from the Rawhide collection and development will continue.
-{{Anchor|feature_freeze}}
-{{Anchor|string_freeze}}
-==== Feature and String Freeze ====
-The tree freeze for Beta marks the [[ReleaseEngineering/FeatureFreezePolicy|  Feature Freeze]]  and [[L10N/Freezes#string|  String Freeze]]  for the next Fedora release.  All planned features for Fedora must be in a testable or finished state or they risk being dropped for the release.  All strings should be finalized for translations to happen before the final release.
-{{Anchor|final_freeze}}
-==== Final Freeze ====
-Roughly 23 days before the final release, the tree will go into the final blocking-freeze mode where it becomes much harder to justify new changes to the collection unless a serious bug-fix or security issue is involved. During the days leading to the final release, the release engineering team is in constant communication with the security team, the documentation maintainers, and the arch maintainers, to ensure that all of the different components required for a successful release are available.  A series of snapshots of Rawhide will be composed as a Pre-releases (or Release Candidates), which will aid in the final QA runs.
-{{Anchor|rawhide_snapshots}}
-==== Rawhide Snapshots ====
-During the course of Fedora development, snapshot releases can be made that are Feature driven, or bugfix driven, or otherwise deemed useful to have but not necessarily scheduled or listed. These also may have less coordinated QA but still be useful for testers to make use of for sampling the release as it goes along. These will likely be [[Distribution/Download/BitTorrent|  BitTorrent]]  only releases. Additionally snapshot releases can be attempted weekly after the Beta has been released. This could give developers a target for getting important bugs fixed or changes in if they know a snapshot is coming up. If the compose fails we just try again next week.
-=== Final Release Checklist ===
 Various tasks need to be accomplished prior to a final Fedora release.  Release Engineering is responsible for many of them, as outlined here.
-==== Release Naming ====
+=== Export Approval ===
-Each Fedora release is given a code name.  Each name has a relationship to the name before it, and the name after it.  We let the community of contributors brainstorm possible names for the release, and run them through Red Hat's legal department for trademark issues.
-We start gathering possible names from Fedora community early in the development cycle of each new release via fedora-devel list and we pass these names through Red Hat Legal for acceptable candidates.  Once list is returned, setup a poll to let the community vote on their favorite name candidate.  Name voting must be finished by the final freeze date for use in fedora-release and various documentation.
-==== Export Approval ====
 As a U.S. based company, Red Hat needs to file for export approval when creating Fedora releases.  Currently this involves giving Red Hat Legal a heads up at the final freeze date, and updating Legal once the final tree has been spun.
-==== Release Announcement ====
+=== Release Announcement ===
-The [[DocsProject|  Fedora Documentation Project]]  prepares release announcements for the final releases.  A bug needs [https://bugzilla.redhat.com/bugzilla/enter_bug.cgi?product=Fedora%20Documentation&op_sys=Linux&target_milestone=---&bug_status=NEW&version=devel&component=release-notes&rep_platform=All&priority=normal&bug_severity=normal&assigned_to=relnotes%40fedoraproject.org&cc=&estimated_time_presets=0.0&estimated_time=0.0&bug_file_loc=http%3A%2F%2F&short_desc=RELNOTES%20-%20Summarize%20the%20release%20note%20suggestion%2Fcontent&comment=Provide%20details%20here.%20%20Do%20not%20change%20the%20blocking%20bug.&status_whiteboard=&keywords=&issuetrackers=&dependson=&blocked=151189&ext_bz_id=0&ext_bz_bug_id=&data=&description=&contenttypemethod=list&contenttypeselection=text%2Fplain&contenttypeentry=&maketemplate=Remember%20values%20as%20bookmarkable%20template&form_name=enter_bug filed]  for this two weeks before the final release date.
+The [[DocsProject|Fedora Documentation Project]] prepares release announcements for the final releases.  A bug needs to be [https://bugzilla.redhat.com/bugzilla/enter_bug.cgi?product=Fedora%20Documentation&op_sys=Linux&target_milestone=---&bug_status=NEW&version=devel&component=release-notes&rep_platform=All&priority=normal&bug_severity=normal&assigned_to=relnotes%40fedoraproject.org&cc=&estimated_time_presets=0.0&estimated_time=0.0&bug_file_loc=http%3A%2F%2F&short_desc=RELNOTES%20-%20Summarize%20the%20release%20note%20suggestion%2Fcontent&comment=Provide%20details%20here.%20%20Do%20not%20change%20the%20blocking%20bug.&status_whiteboard=&keywords=&issuetrackers=&dependson=&blocked=151189&ext_bz_id=0&ext_bz_bug_id=&data=&description=&contenttypemethod=list&contenttypeselection=text%2Fplain&contenttypeentry=&maketemplate=Remember%20values%20as%20bookmarkable%20template&form_name=enter_bug filed] for this two weeks before the final release date.
-==== Mirror List Files ====
+=== Mirror List Files ===
 A new set of mirror list files need to be created for the new release.  Email [mailto:mirror-admin@fedoraproject.org Fedora Mirror Admins]  to have these files created.  These should be created at the Final Freeze point but may redirect to Rawhide until final bits have been staged.
-==== Update Tool ====
+=== Update Tool ===
-The Fedora Update tool will need to be updated to handle update requests for the new release.  This needs to happen at the CVS Branching point, currently coordinated with the Final Freeze.
+The Fedora Update tool will need to be updated to handle update requests for the new release. Updates for the new release should start to be pushed a few days before the release date.  This will allow for ensuring the update process will work, and to get eyes on the update bits before the rest of the world gets them.  This will also allow those who were early adopters of Fedora to get updates after the final bits have been composed into a release.
-Updates for the new release should start to be pushed a few days before the release date.  This will allow for ensuring the update process will work, and to get eyes on the update bits before the rest of the world gets them.  This will also allow those who were early adopters of Fedora to get updates after the final bits have been composed into a release.
+<!--
+*** ADAMW 2014-09 ***
+This entire section is clearly heavily outdated, but I don't know if anything should replace it, so I felt better about commenting it out than burning it down.
-==== CVS Branching ====
+=== Git Branching ===
-===== Early (optional) CVS branching =====
+==== Early (optional) git branching ====
-Right after Beta release or sometime between Beta and Pre-release, we allow for early branching of software. This allows developers to check in new features and otherwise unstable changes that would not be suitable to introduce to the current release.
+At the public release of the Alpha release we allow for early branching of packages for the next release. This allows developers to check in new features and otherwise unstable changes that would not be suitable to introduce to the current release.
-===== Mass CVS branching =====
+==== Mass git branching ====
-At the final freeze, a "branch" in CVS is made, for everything that hasn't already been branched, to reflect the release about to be made.  These branches are made from the current devel/ branch and retain all the history thereof.  The CVS tree will be inaccessible during the branching period which should only take a few hours.  The CVS admins will perform this task.  Any builds from the branch would be held in an updates candidate tag to be potential updates for after the release, or to be pulled in to the release by request.  Builds from devel/ would be tagged for the next release tag to show up in Rawhide once the next development cycle starts.
+At the final freeze, a "branch" in git is made, for everything that hasn't already been branched, to reflect the release about to be made.  These branches are made from the current devel/ branch and retain all the history thereof.  The git tree will be inaccessible during the branching period which should only take a few hours.  The git admins will perform this task.  Any builds from the branch would be held in an updates candidate tag to be potential updates for after the release, or to be pulled in to the release by request.  Builds from devel/ would be tagged for the next release tag to show up in Rawhide once the next development cycle starts.
 '''Note''': Requests for new F-(X-2) packages when F-X goes gold will not be accepted. For example, when Fedora 10 is released, no new package requests for Fedora 8 will be accepted.
+-->
 == Release Composing ==
-Fedora “releases” can be built by anyone with a fast machine of proper arch and access to a package repository.  All of the tools necessary to build a release are available from the package repository. These tools aim to provide a consistent way to build Fedora releases. A complete release can actually be built with only a couple commands, including the creation of ISO images suitable for burning to CD/DVD Media, Live Images, and a network install directory. These commands are named pungi and livecd-creator.
+Fedora “releases” can be built by anyone with a fast machine of proper arch and access to a package repository.  All of the tools necessary to build a release are available from the package repository. These tools aim to provide a consistent way to build Fedora releases. A complete release can actually be built with only a couple commands, including the creation of network install images, offline install images ('DVDs'), live images, disk images, install repositories, [[FedUp]] upgrade images, and other bits. These commands are named pungi and livecd-creator.
 === Pungi ===
-[https://fedorahosted.org/pungi Pungi]  is the tool used to compose Fedora releases.  It requires being ran in a chroot of the package set that it is composing.  This ensures that the correct userland tools are used to match up with the kernel that will be used to perform the installation.  It uses a comps file + yum repos to gather packages for the compose.  The Fedora Buildsystem provides a way to run tasks within chroots on the various arches, as well as the ability to produce yum repos of packages from specific collections.
+[https://fedorahosted.org/pungi Pungi]  is the tool used to compose Fedora releases.  It requires being ran in a chroot of the package set that it is composing.  This ensures that the correct userland tools are used to match up with the kernel that will be used to perform the installation.  It uses a comps file + yum repos to gather packages for the compose.  The [[Koji]] build system provides a way to run tasks within chroots on the various arches, as well as the ability to produce yum repos of packages from specific collections.
 === Livecd-creator ===
-Livecd-creator is part of the [[FedoraLiveCD | livecd-tools]] package in Fedora and it is used to compose the live images, both CD's and DVD's as part of the Fedora release. It is also used to compose many of the custom spins or variants of Fedora.
+Livecd-creator is part of the [[FedoraLiveCD | livecd-tools]] package in Fedora and it is used to compose the live images as part of the Fedora release. It is also used to compose many of the custom [https://spins.fedoraproject.org Spins] or variants of Fedora.
 == Distribution ==
-Once a compose has been completed, the composed tree of ISOs, install trees, and Live Images needs to be syncronized with the Fedora mirror system.  More details about this are forthcoming as the Fedora Mirrorsystem is built up.  Various ISOs are also offered via BitTorrent as an alternative method of downloading.
+Once a compose has been completed, the composed tree of release media, installation trees, and frozen [[Repositories]] needs to be synchronized with the Fedora mirror system. [[MirrorManager]] has some more details on the mirror system. Many of the images are also offered via BitTorrent as an alternative method of downloading.
 === Download Mirrors ===
@@ Line 134: / Line 78: @@
 Depends on the Fedora Mirror System and infrastructure to populate them privately.
-=== Bittorrent ===
+=== BitTorrent ===
-Bittorent is currently served by torrent.fedoraproject.org.  Isos are added to the system via this [[Infrastructure/SOP/TorrentRelease|  Standard Operating Procedure]] .
+BitTorrent is currently served by http://torrent.fedoraproject.org. Images are added to the system via this [[Infrastructure/SOP/TorrentRelease|Standard Operating Procedure]].
 == Acknowledgements ==
-This document was influenced by [http://www.freebsd.org/doc/en_US.ISO8859-1/articles/releng/article.html release engineering documents]  from [http://freebsd.org FreeBSD] .
+This document was influenced by [http://www.freebsd.org/doc/en_US.ISO8859-1/articles/releng/article.html release engineering documents] from [http://freebsd.org FreeBSD].
-<!--Important pages can be categorized by putting CategoryFoo at the bottom of the page under a horizontal rule. Replace Foo with a category name such as Documentation.
--->
 [[Category:Release Engineering]]

Search

ReleaseEngineering/Overview: Difference between revisions