Latest revision as of 16:45, 12 September 2019

Retire Python 2

Summary

The python2 package and all its subpackages will be removed from Fedora 32. A legacy python27 package for developers and users will be provided. All packages in Fedora that need Python 2 to run will be removed from Fedora 32 regardless of their dependencies. All packages in Fedora that need Python 2 to build will be removed from Fedora 32 regardless of their dependencies. Exceptions can be granted by FESCo.

Owner

Name: Miro Hrončok
Email: <mhroncok@redhat.com> <python-devel@lists.fedoraproject.org>
Release notes owner:

Current status

Targeted release: Fedora 32
Last updated: 2019-09-12
Tracker bug: #1708725
Release notes tracker: #338

Detailed Description

Python 2 is unsupported upstream since 2020-01-01. Packages dependent on Python 2 are being removed from Fedora for several releases already:

Now, the Python maintainers have decided to pull the plug. The python2 package and all its subpackages will be retired (read: removed) from Fedora 32 (Rawhide) as soon as Fedora 31 is branched (in fact we'll wait until after the Changes/Python3.8 side tag is merged).

All packages depending on any python2 package will be removed. The removal starts mid November 2019. Broken dependencies will not stop the removals. Packages that Fail to Build From Source and prevent to remove Python 2 subpackages may end up with broken dependencies, in cases where it is not desired, those packages will be retired instead.

The rules also apply to modules built for Fedora 32+.

The package removal will be executed in an automated fashion.

Removed packages that would block the upgrades to Fedora 32 will be obsoleted from fedora-obsolete-packages.

The python27 package

Similarly to existing python36, python37 etc. packages, a python27 package will be created. This package is intended for Python developers who still need to support the legacy version of Python. This package is intended for users, who still need to use some software depending on the legacy version of Python. This package is not intended for other Fedora packages to be depended upon.

The python27 package has several drawbacks compared to the original python2 package:

it is "flat" - there are no subpackages, everything lives in one package
there is no debug build (previously available as python2-debug)
there is no /usr/bin/python (note: there might be already the case before this change)
any special backwards compatible Provides are removed (this package is not intended to be depended upon)

FESCo exceptions

We realize that there are some packages whose removal could seriously hurt Fedora. FESCo can grant exceptions for packages to use the python27 as a runtime or build dependency.

The package maintainer is responsible to check the entire dependency chain and they need to request exceptions for the entire list of packages. For example, when seeking exception for the chromium package, the request should contain python-psutil and other dependent packages. (Yes, this is tedious. Maintaining a Python 2 dependent package is a burden.)

The exception request must include a plan for migrating to Python 3.

Any non-essential dependency must be dropped. That includes optional dependencies, test dependencies, optional subpackages etc.

Package that fail to get an exception when the removal starts (see above) will be removed. Their importance for Fedora Release Engineering, Fedora Infrastructure or any other body will not be automagically respected; every package that needs Python 2 needs an exception.

The change owners will send regular reminders to the package owners.

Benefit to Fedora

Python 2 is past upstream End of Life since 2020-01-01. This changes is generally crafted in a way that:

it leaves Python developers an option to use it in case they still need to support it
it leaves Fedora users an option to use it in case they still need it to run their (3rd party) software
it leaves Fedora packagers an option to keep using it (complicated, but possible)

While:

it removes Python 2 software from Fedora that was only preserved so far by inaction

Using Python 2 is dangerous. While the Fedora Python maintainers will try to fix as many security bugs as possible, without the upstream involvement this will be hard.

Python 2 is deprecated since Fedora 30. This change moves Python 2 from second class citizen to third class citizen.

Scope

Proposal owners:
- retire python2 (done)
- introduce python27 (done)
- remove all python2 dependent packages that do not have FESCo exceptions (mid November 2019)
- obsolete removed packages that break the upgrade path via fedora-obsolete-packages

Other developers:
- remove their python2 dependent packages without exceptions
- get exceptions if needed
- fix broken dependencies

Release engineering: #8306 (a check of an impact with Release Engineering is needed)
- List of deliverables: none

Policies and guidelines: Python 2 packaging is against the guidelines since Fedora 30. Python 2 packaging guidelines will be removed from Python Appendix (unless the FPC wants to keep them around until F31 EOL).

Trademark approval: not needed for this Change

Upgrade/compatibility impact

The majority of removed packages will be obsoleted and removed on upgrade.

Users needing Python 2 libraries will not find these packaged as RPMs. They may install upstream versions using pip and virtualenv.

How To Test

Try to update Fedora 30 or 31 to 32. No python2 packages should block the upgrade. Try to run Python 2 software via the python27 package.

User Experience

There will be close to zero Python 2 RPMs in Fedora repos. Users are encouraged to switch to Python 3 and/or use Python 2 virtual environments and pip for development.

Dependencies

Ideally, all programs that use python2 would be switched to use python3. Although we don't expect everything to be switched over, as much as possible should be, so that the ripped remaining python2 set is small as possible.

Contingency Plan

Contingency mechanism:
- In case of serious issues, FESCo can issue a general exception for packages that would otherwise prevent Fedora 32 from being composed.
- If someone steps up to maintain Python 2 (including the full ecosystem of packages now in Fedora), they can decide to discontinue removing packages, revert this Change, or come up with another plan. (Note that in this case, current maintainers will most likely orphan many fundamental python2 packages.)
Contingency deadline: Fedora 32 Beta
Blocks release? in theory it should not, in practice, it may break the release and hence it will block it until fixed
Blocks product? all of them?

Documentation

This page should serve as the documentation.

Release Notes

TBD.

@@ Line 19: / Line 19: @@
 <!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->
+= Retire Python 2 =
-<!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->
-= Retire Python 2 (and let it all burn) =
 == Summary ==
@@ Line 27: / Line 25: @@
 Note that motivation for the change should be in the Motivation section below, and this part should answer the question "What?" rather than "Why?". -->
-The {{package|python2}} package and all it's subpackages will be removed from Fedora 32.
+The {{package|python2}} package and all its subpackages will be removed from Fedora 32.
 A legacy {{package|python27}}  package for developers and users will be provided.
 All packages in Fedora that need Python 2 to run will be removed from Fedora 32 regardless of their dependencies.
@@ Line 61: / Line 59: @@
 CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
 -->
-* Tracker bug: <will be assigned by the Wrangler>
+* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1708725 #1708725]
+* Release notes tracker: [https://pagure.io/fedora-docs/release-notes/issue/338 #338]
 == Detailed Description ==
 <!-- Expand on the summary, if appropriate.  A couple sentences suffices to explain the goal, but the more details you can provide the better. -->
+Python 2 is unsupported upstream since 2020-01-01. Packages dependent on Python 2 are being removed from Fedora for several releases already:
+* [[Changes/Mass_Python_2_Package_Removal|Fedora 30 Mass Python 2 Package Removal]]
+* [[Changes/F31_Mass_Python_2_Package_Removal|Fedora 31 Mass Python 2 Package Removal]]
+Now, the Python maintainers have decided to pull the plug. The {{package|python2}} package and all its subpackages will be retired (read: removed) from Fedora 32 (Rawhide) as soon as Fedora 31 is branched (in fact we'll wait until after the [[Changes/Python3.8]] side tag is merged).
+All packages depending on any python2 package will be removed. '''The removal starts mid November 2019.'''
+Broken dependencies will not stop the removals.
+Packages that Fail to Build From Source and prevent to remove Python 2 subpackages may end up with broken dependencies,
+in cases where it is not desired, those packages will be retired instead.
+The rules also apply to modules built for Fedora 32+.
+The package removal will be executed in an automated fashion.
+Removed packages that would block the upgrades to Fedora 32 will be obsoleted from {{package|fedora-obsolete-packages}}.
+=== The python27 package ===
+Similarly to existing {{package|python36}}, {{package|python37}} etc. packages, a {{package|python27}} package will be created.
+This package is intended for Python developers who still need to support the legacy version of Python.
+This package is intended for users, who still need to use some software depending on the legacy version of Python.
+This package is not intended for other Fedora packages to be depended upon.
+The {{package|python27}} package has several drawbacks compared to the original {{package|python2}} package:
+* it is "flat" - there are no subpackages, everything lives in one package
+* there is no debug build (previously available as {{package|python2-debug}})
+* there is no <code>/usr/bin/python</code> (note: there might be already the case before this change)
+* any special backwards compatible Provides are removed (this package is not intended to be depended upon)
+=== FESCo exceptions ===
+We realize that there are some packages whose removal could seriously hurt Fedora. FESCo can grant exceptions for packages to use the {{package|python27}} as a runtime or build dependency.
+The package maintainer is responsible to check the entire dependency chain and they need to request exceptions for the entire list of packages. For example, when seeking exception for the {{package|chromium}} package, the request should contain {{package|python-psutil}} and other dependent packages. (Yes, this is tedious. Maintaining a Python 2 dependent package is a burden.)
+The exception request must include a plan for migrating to Python 3.
+Any non-essential dependency must be dropped. That includes optional dependencies, test dependencies, optional subpackages etc.
+Package that fail to get an exception when the removal starts (see above) will be removed. Their importance for Fedora Release Engineering, Fedora Infrastructure or any other body will not be automagically respected; every package that needs Python 2 needs an exception.
+The change owners will send regular reminders to the package owners.
 == Benefit to Fedora ==
@@ Line 96: / Line 141: @@
       https://fedoraproject.org/wiki/Changes/perl5.26 (major upgrade to a popular software stack, visible to users of that stack)
 -->
+Python 2 is past upstream End of Life since 2020-01-01. This changes is generally crafted in a way that:
+* it leaves Python developers an option to use it in case they still need to support it
+* it leaves Fedora users an option to use it in case they still need it to run their (3rd party) software
+* it leaves Fedora packagers an option to keep using it (complicated, but possible)
+While:
+* it removes Python 2 software from Fedora that was only preserved so far by inaction
+Using Python 2 is dangerous. While the Fedora Python maintainers will try to fix as many security bugs as possible, without the upstream involvement this will be hard.
+Python 2 is deprecated since Fedora 30. This change moves Python 2 from second class citizen to third class citizen.
 == Scope ==
 * Proposal owners:
+** retire {{package|python2}} (done)
+** introduce {{package|python27}} (done)
+** remove all {{package|python2}} dependent packages that do not have FESCo exceptions (mid November 2019)
+** obsolete removed packages that break the upgrade path via {{package|fedora-obsolete-packages}}
 <!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
-* Other developers: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* Other developers: <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+** remove their {{package|python2}} dependent packages without exceptions
+** get exceptions if needed
+** fix broken dependencies
 <!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
-* Release engineering: [https://pagure.io/releng/issues #Releng issue number] (a check of an impact with Release Engineering is needed) <!-- REQUIRED FOR SYSTEM WIDE AS WELL AS FOR SELF CONTAINED CHANGES -->
+* Release engineering: [https://pagure.io/releng/issue/8306 #8306] (a check of an impact with Release Engineering is needed) <!-- REQUIRED FOR SYSTEM WIDE AS WELL AS FOR SELF CONTAINED CHANGES -->
 <!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)?  Is a mass rebuild required?  include a link to the releng issue.
 The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing, and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->
-** [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+** [[Fedora_Program_Management/ReleaseBlocking/Fedora{{FedoraVersionNumber|next}}|List of deliverables]]: none <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 <!-- Please check the list of Fedora release deliverables and list all the differences the feature brings -->
-* Policies and guidelines: N/A (not a System Wide Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* Policies and guidelines: Python 2 packaging is against the guidelines since Fedora 30. Python 2 packaging guidelines will be removed from [https://docs.fedoraproject.org/en-US/packaging-guidelines/Python_Appendix/ Python Appendix] (unless the FPC wants to keep them around until F31 EOL).<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 <!-- Do the packaging guidelines or other documents need to be updated for this feature?  If so, does it need to happen before or after the implementation is done?  If a FPC ticket exists, add a link here. -->
-* Trademark approval: N/A (not needed for this Change)
+* Trademark approval: not needed for this Change
 <!-- If your Change may require trademark approval (for example, if it is a new Spin), file a ticket ( https://fedorahosted.org/council/ ) requesting trademark approval from the Fedora Council. This approval will be done via the Council's consensus-based process. -->
@@ Line 120: / Line 186: @@
 <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-N/A (not a System Wide Change)
+The majority of removed packages will be obsoleted and removed on upgrade.
+Users needing Python 2 libraries will not find these packaged as RPMs. They may install upstream versions using pip and virtualenv.
 == How To Test ==
@@ Line 138: / Line 207: @@
 <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-N/A (not a System Wide Change)
+Try to update Fedora 30 or 31 to 32. No python2 packages should block the upgrade.
+Try to run Python 2 software via the {{package|python27}} package.
 == User Experience ==
@@ Line 151: / Line 221: @@
   - Green has been scientifically proven to be the most relaxing color. The move to a default background color of green with green text will result in Fedora users being the most relaxed users of any operating system.
 -->
+There will be close to zero Python 2 RPMs in Fedora repos. Users are encouraged to switch to Python 3 and/or use Python 2 virtual environments and pip for development.
 == Dependencies ==
@@ Line 156: / Line 228: @@
 <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-N/A (not a System Wide Change)
+Ideally, all programs that use python2 would be switched to use python3. Although we don't expect everything to be switched over, as much as possible should be, so that the ripped remaining python2 set is small as possible.
 == Contingency Plan ==
 <!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
-* Contingency mechanism: (What to do?  Who will do it?) N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* Contingency mechanism:
+** In case of serious issues, FESCo can issue a general exception for packages that would otherwise prevent Fedora 32 from being composed.
+** If someone steps up to maintain Python 2 (including the full ecosystem of packages now in Fedora), they can decide to discontinue removing packages, revert this Change, or come up with another plan. (Note that in this case, current maintainers will most likely orphan many fundamental python2 packages.)
+<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 <!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
-* Contingency deadline: N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* Contingency deadline: Fedora 32 Beta <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 <!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
-* Blocks release? N/A (not a System Wide Change), Yes/No <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+* Blocks release? in theory it should not, in practice, it may break the release and hence it will block it until fixed <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-* Blocks product? product <!-- Applicable for Changes that blocks specific product release/Fedora.next -->
+* Blocks product? all of them? <!-- Applicable for Changes that blocks specific product release/Fedora.next -->
 == Documentation ==
@@ Line 172: / Line 247: @@
 <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-N/A (not a System Wide Change)
+This page should serve as the documentation.
+=== See Also ===
+* [https://www.python.org/doc/sunset-python-2/ Sunsetting Python 2]
 == Release Notes ==
@@ Line 181: / Line 260: @@
 -->
-[[Category:ChangePageIncomplete]]
+TBD.
+[[Category:ChangeAcceptedF32]]
 <!-- When your change proposal page is completed and ready for review and announcement -->
 <!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->

Search

Changes/RetirePython2: Difference between revisions