From Fedora Project Wiki
(Remove F21 category, it's multi-release but can be confusing)
(30 intermediate revisions by 3 users not shown)
Line 1: Line 1:
= Python 3 as Default =
= Python 3 as the Default Implementation =


== Summary ==
== Summary ==
Up until now, Fedora has used Python 2 as a default Python implementation. This change proposes switching to Python 3. The term "switching" is explained in detail in Scope section.
Up until now, Fedora has used Python 2 as the default Python implementation. This change proposes switching to Python 3. The details of the term "switching" are explained thoroughly in the [[#Scope|Scope]] section.


== Owner ==
== Owner ==
<!--  
<!--
For change proposals to quality as self-contained, owners of all affected packages need to be included here. Alternatively, a SIG can be listed as an owner if it owns all affected packages.  
For change proposals to quality as self-contained, owners of all affected packages need to be included here. Alternatively, a SIG can be listed as an owner if it owns all affected packages.
This should link to your home wiki page so we know who you are.  
This should link to your home wiki page so we know who you are.
-->
-->
* Name: [[User:Bkabrda|Slavek Kabrda]]
* Name: [[User:Bkabrda|Slavek Kabrda]]
* Email: bkabrda@redhat.com
* Email: bkabrda@redhat.com
* Name: [[User:Mstuchli|Matej Stuchlik]]
* Email: mstuchli@redhat.com
* Name: [[User:Churchyard|Miro Hroncok]]
* Email: mhroncok@redhat.com
* Release notes owner: <!--- To be assigned by docs team [[User:FASAccountName| Release notes owner name]] <email address> -->
* Release notes owner: <!--- To be assigned by docs team [[User:FASAccountName| Release notes owner name]] <email address> -->
<!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
<!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
Line 17: Line 21:


== Current status ==
== Current status ==
* Targeted release: [[Releases/22 | Fedora 22 ]]  
* Targeted release: [[Releases/22 | Fedora 22 ]]
* Last updated: 10-03-2013
* Last updated: 10-03-2013
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page  
<!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page
Bugzilla states meaning as usual:
Bugzilla states meaning as usual:
NEW -> change proposal is submitted and announced
NEW -> change proposal is submitted and announced
Line 27: Line 31:
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
-->
-->
* Tracker bug: <will be assigned by the Wrangler> (we already created a tracker bug: https://bugzilla.redhat.com/show_bug.cgi?id=1014209)
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1076441 #1076441] (we already created a tracker bug: https://bugzilla.redhat.com/show_bug.cgi?id=1014209)


== Detailed Description ==
== Detailed Description ==
Python 3 is the next generation of Python programming language. It is currently mature and stable - 3.0 was released in December 2008, current latest stable version is 3.3.2 released in May 2013. The reasons why to switch to Python 3 as a default are mentioned in Benefit to Fedora section, possible issues in Scope. This feature assumes that DNF will be the default package manager in F22, as Yum doesn't and never will work with Python 3. If DNF doesn't make it to Fedora 22, this feature has to be postponed accordingly.
Python 3 is the next generation of Python programming language. It is currently mature and stable, since it has been under active development for five years - version 3.0 was released in December 2008, current latest stable version is 3.3.2 released in May 2013. The main reason to switch to Python 3 as the default implementation is that Python 2 is in maintenance mode, thus only bugfixes and security fixes are accepted upstream. Further reasons are mentioned in the [[#Benefit to Fedora|Benefit to Fedora]] section.
For this Change to be carried out successfully, it is necessary that the key packages in the Fedora software stack be ported to Python 3. These are parts of the minimal buildroot, the default package manager, programs present on the LiveCD etc. More information on the packages involved can be found in [[#Dependencies|Dependencies]]. While porting of some packages is rather trivial, other packages need significant amount of work to get rid of the Python 2 dependence.


== Benefit to Fedora ==
== Benefit to Fedora ==
<!-- What is the benefit to the platform?  If this is a major capability update, what has changed?  If this is a new functionality, what capabilities does it bring? Why will Fedora become a better distribution or project because of this proposal?-->
<!-- What is the benefit to the platform?  If this is a major capability update, what has changed?  If this is a new functionality, what capabilities does it bring? Why will Fedora become a better distribution or project because of this proposal?-->
Python 2.7 (latest Python 2 release, which we also have in Fedora) is currently in maintenance mode only, which means upstream only accepts bugfixes and security fixes, but no new features are being implemented. According to upstream, Python 2.7 is the last release of Python 2 (http://www.python.org/dev/peps/pep-0404/). Support for Python 2 is guaranteed by upstream until May 2015 (http://www.python.org/dev/peps/pep-0373/#maintenance-releases), then it may continue for some time or it may not. Python 3, on the other hand, is actively developed and new features are being added every release.
Python 2.7 (latest Python 2 release, which we also have in Fedora) is currently in maintenance mode only, which means upstream only accepts bugfixes and security fixes, but no new features are being implemented. According to upstream, Python 2.7 is the last release of Python 2 (http://www.python.org/dev/peps/pep-0404/). Support for Python 2 is guaranteed by upstream until May 2015 (http://www.python.org/dev/peps/pep-0373/#maintenance-releases), then it may continue for some time, or it may not. Python 3, on the other hand, is actively developed and new features are being added every release. Moreover, there is currently no end of support date for Python 3.


Fedora already has Python 3 stack, that is parallel to Python 2 stack. The are few benefits of switching the "primary" Python stack:
Fedora already has Python 3 stack that is parallel to Python 2 stack. The are several benefits of switching the "primary" Python stack:
* Getting upstream support is not limited by time.
* Getting upstream support for default system version will not be limited by time.
* Our system tools will be able to switch to Python 3, drop Python 2 support and use new features of Python 3.
* Our system tools will be able to switch to Python 3, drop the burden of Python 2 support and use new features of Python 3.
* As a distribution that stays close to upstream, Fedora should help Python community go forward by contributing patches and working closely with upstreams to get this accomplished. Thus this Change is meant to benefit not only Fedora, but also broad Python community.
* As a distribution that stays close to upstream, Fedora should help Python community go forward by contributing patches and working closely with upstreams to get this accomplished. Thus this Change is meant to benefit not only Fedora, but also broader Python community.
* Pushing Python 3 as a default in Fedora should improve perception of Python 3 as a mature language and speed up adoption by upstreams.
* Switching to Python 3 as a default will once again push Fedora to stay as close to upstream as possible, highlighting the "Features" and "First" (although, to be honest, Arch Linux was first in this...)


== Scope ==
== Scope ==
<!-- What work do the developers have to accomplish to complete the change in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
<!-- What work do the developers have to accomplish to complete the change in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
The main goal is switching to Python 3 as a default, in which state:
The main goal is switching to Python 3 as a default, in which state:
* DNF is the default installed instead of Yum, which only works with Python 2
* DNF is the default package manager instead of Yum, which only works with Python 2
* Python 3 is the only Python in minimal buildroot
* Python 3 is the only Python implementation in the minimal buildroot
* Python 3 is the only Python on LiveCD
* Python 3 is the only Python implementation on the LiveCD
* Anaconda and all its dependencies run on Python 3
* Anaconda and all of its dependencies run on Python 3
* cloud-init and all its dependencies run on Python 3
* cloud-init and all of its dependencies run on Python 3
This will also require revisiting Python guidelines (broader discussion with community and FPC approval - TBD). The result of the discussion will reflect in this feature in further instructions for Fedora packagers.
(see [[#Dependencies|Dependencies]] for the list of packages that need to be ported)


Work needed to be done:
This will also require revisiting Python guidelines (broader discussion with community and FPC approval - TBD). The result of the discussion will reflect in this Change in further instructions for Fedora packagers.


There are basically two types of packages that need to undergo the conversion:
* Python extension modules and libraries that provide Python bindings - assuming that there is upstream support, these can receive python3- subpackage anytime without any damage to Fedora; we can then just utilize this subpackage when switching to Python 3 (instead of using current python- subpackage).
* Packages that build with some sort of "embedded Python support", like gdb, or Rhythmbox with its plugins. In these cases, it makes no sense to do a python3- subpackage, since the whole package would need to be duplicated (e.g. python3-gdb). These packages should be built with Python 3 in Fedora 22 rawhide as soon as possible.
=== Work in Fedora 21 Timeframe ===
* Proposal owners:
* Proposal owners:
<!-- What work do the feature owners have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
** Discussing changes in Python packaging guidelines with Fedora community and FPC
** Help various upstreams with migration to Python 3 (already in progress for some time)
** Helping upstreams with porting to Python 3
** Discuss changes in Python packaging guidelines with Fedora community and FPC
** Introducing python3- packages where appropriate, testing packages that only build with Python once (e.g. gdb, Rhythmbox)
 
* Other developers:
** Hopefully the same as proposal owners.
 
* Release engineering:
** Nothing.


* Policies and guidelines:
** As mentioned above, this will require a discussion with community and FPC and preparation of changes to Python packaging guidelines. The changes related to the actual switch should however be merged in F22 timeframe and only if the switch is successful.
=== Work in Fedora 22 Timeframe ===
* Proposal owners:
** Continue the work from F21 timeframe
** Modify comps accordingly
** Apply the changes to Python packaging guidelines


* Other developers:
* Other developers:
<!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
** Introduce python3- subpackages where appropriate, build against Python 3 if the package supports it
** Packages who own a package that uses Python in any way should make sure that it works with Python 3.
*** If the package works with Python 3, they should add python3- subpackage where appropriate.
*** For some packages that build with Python, like gdb or Rhythmbox, doing python3- subpackages makes no sense. Packagers should only make sure that the packages work with Python 3. There should a single point, at which all these packages will be rebuilt with Python 3, possibly in a Koji side tag.
*** If the package doesn't work with Python 3, packagers should encourage upstreams and send patches.


* Release engineering:
* Release engineering:
<!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)?  Is a mass rebuid required?  If a rel-eng ticket exists, add a link here.  -->
** Nothing
** We will use a side tag in Koji and merge it to rawhide only if everything works as expected.


* Policies and guidelines:
* Policies and guidelines:
<!-- Do the packaging guidelines or other documents need to be updated for this feature?  If so, does it need to happen before or after the implementation is done?  If a FPC ticket exists, add a link here. -->
** Apply the changes to Python guidelines if the switch is achieved
** As mentioned above, this will require a discussion with community and FPC and alteration of Python packaging guidelines.


== Upgrade/compatibility impact ==
== Upgrade/compatibility impact ==
Line 79: Line 97:


<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
The Python 2 stack will stay in Fedora, it will just not be the default one. Depending on the modifications done to Python packaging guidelines, this probably means that Python 2 packages will stay the way they are and default tools will drag in python3- dependencies. Upstream recommends, that /usr/bin/python should point to Python 2 runtime for the time being, so if we go with that, there shouldn't be any serious compatibility impact:
The Python 2 stack will stay in Fedora, it will just not be the default one. Depending on the modifications done to Python packaging guidelines, this probably means that Python 2 packages will stay the way they are and default tools will drag in python3- dependencies. Upstream recommends that /usr/bin/python point to Python 2 runtime for the time being, so if we go with that, there shouldn't be any serious compatibility impact:
* Users will still be able to install Python 2 packages
* Users will still be able to install Python 2 packages
* Both Python 2 and 3 stacks will still live in parallel
* Both Python 2 and 3 stacks will still live in parallel
Libraries that are built with Python only once (like gdb) may force users to rewrite their custom scripts and plugins (see e.g. https://bugzilla.redhat.com/show_bug.cgi?id=1014549#c3).


== How To Test ==
== How To Test ==
<!-- This does not need to be a full-fledged document. Describe the dimensions of tests that this change implementation is expected to pass when it is done.  If it needs to be tested with different hardware or software configurations, indicate them.  The more specific you can be, the better the community testing can be.  
<!-- This does not need to be a full-fledged document. Describe the dimensions of tests that this change implementation is expected to pass when it is done.  If it needs to be tested with different hardware or software configurations, indicate them.  The more specific you can be, the better the community testing can be.


Remember that you are writing this how to for interested testers to use to check out your change implementation - documenting what you do for testing is OK, but it's much better to document what *I* can do to test your change.
Remember that you are writing this how to for interested testers to use to check out your change implementation - documenting what you do for testing is OK, but it's much better to document what *I* can do to test your change.
Line 99: Line 118:


<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
TBD
* No special hardware is needed.
* Install Fedora 22 in mock/virtual machine and test if everything still works, most importantly system tools like DNF or Firewalld.
* Test installing the system with Anaconda running on Python 3.


== User Experience ==
== User Experience ==
<!-- If this change proposal is noticeable by its target audience, how will their experiences change as a result?  Describe what they will see or notice. -->
<!-- If this change proposal is noticeable by its target audience, how will their experiences change as a result?  Describe what they will see or notice. -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
Users shouldn't notice any change in behaviour, except from installation of python3- packages by default.
Users shouldn't notice any changes, except that packages in minimal buildroot and on LiveCD will be python3-, not python-. Anaconda deps will be python3- as well. /usr/bin/python will still point to Python 2 and depending on result of discussions with FPC, "yum install python-foo" will still install Python 2 version of the package.


== Dependencies ==
== Dependencies ==
Line 114: Line 135:
== Contingency Plan ==
== Contingency Plan ==
<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
* Contingency mechanism: (What to do?  Who will do it?) Don't merge the koji side-tag into rawhide, revert changes in dist-git.
* Contingency mechanism: None needed. Packages that will be ready will be built with Python 3, the rest will be ported in next release.
<!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
<!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
* Contingency deadline: Development freeze
* Contingency deadline: Software string freeze
<!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
<!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
* Blocks release? No
* Blocks release? No
Line 125: Line 146:
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
http://docs.python.org/dev/howto/pyporting.html
http://docs.python.org/dev/howto/pyporting.html
http://docs.python.org/3/howto/cporting.html
http://docs.python.org/3/howto/cporting.html
https://wiki.gnome.org/PyGObject/IntrospectionPorting
https://wiki.gnome.org/PyGObject/IntrospectionPorting


== Release Notes ==
== Release Notes ==
<!-- The Fedora Release Notes inform end-users about what is new in the release.  Examples of past release notes are here: http://docs.fedoraproject.org/release-notes/ -->
<!-- The Fedora Release Notes inform end-users about what is new in the release.  Examples of past release notes are here: http://docs.fedoraproject.org/release-notes/ -->
<!-- The release notes also help users know how to deal with platform changes such as ABIs/APIs, configuration or data file formats, or upgrade concerns.  If there are any such changes involved in this change, indicate them here.  A link to upstream documentation will often satisfy this need.  This information forms the basis of the release notes edited by the documentation team and shipped with the release.  
<!-- The release notes also help users know how to deal with platform changes such as ABIs/APIs, configuration or data file formats, or upgrade concerns.  If there are any such changes involved in this change, indicate them here.  A link to upstream documentation will often satisfy this need.  This information forms the basis of the release notes edited by the documentation team and shipped with the release.


Release Notes are not required for initial draft of the Change Proposal but has to be completed by the Change Freeze.  
Release Notes are not required for initial draft of the Change Proposal but has to be completed by the Change Freeze.
-->
-->


[[Category:ChangePageIncomplete]]
[[Category:ChangeAcceptedF22]]
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- When your change proposal page is completed and ready for review and announcement -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
<!-- The Wrangler announces the Change to the devel-announce list and changes the category to Category:ChangeAnnounced (no action required) -->  
<!-- The Wrangler announces the Change to the devel-announce list and changes the category to Category:ChangeAnnounced (no action required) -->
<!-- After review, the Wrangler will move your page to Category:ChangeReadyForFesco... if it still needs more work it will move back to Category:ChangePageIncomplete-->
<!-- After review, the Wrangler will move your page to Category:ChangeReadyForFesco... if it still needs more work it will move back to Category:ChangePageIncomplete-->


Revision as of 07:29, 7 April 2014

Python 3 as the Default Implementation

Summary

Up until now, Fedora has used Python 2 as the default Python implementation. This change proposes switching to Python 3. The details of the term "switching" are explained thoroughly in the Scope section.

Owner

Current status

Detailed Description

Python 3 is the next generation of Python programming language. It is currently mature and stable, since it has been under active development for five years - version 3.0 was released in December 2008, current latest stable version is 3.3.2 released in May 2013. The main reason to switch to Python 3 as the default implementation is that Python 2 is in maintenance mode, thus only bugfixes and security fixes are accepted upstream. Further reasons are mentioned in the Benefit to Fedora section. For this Change to be carried out successfully, it is necessary that the key packages in the Fedora software stack be ported to Python 3. These are parts of the minimal buildroot, the default package manager, programs present on the LiveCD etc. More information on the packages involved can be found in Dependencies. While porting of some packages is rather trivial, other packages need significant amount of work to get rid of the Python 2 dependence.

Benefit to Fedora

Python 2.7 (latest Python 2 release, which we also have in Fedora) is currently in maintenance mode only, which means upstream only accepts bugfixes and security fixes, but no new features are being implemented. According to upstream, Python 2.7 is the last release of Python 2 (http://www.python.org/dev/peps/pep-0404/). Support for Python 2 is guaranteed by upstream until May 2015 (http://www.python.org/dev/peps/pep-0373/#maintenance-releases), then it may continue for some time, or it may not. Python 3, on the other hand, is actively developed and new features are being added every release. Moreover, there is currently no end of support date for Python 3.

Fedora already has Python 3 stack that is parallel to Python 2 stack. The are several benefits of switching the "primary" Python stack:

  • Getting upstream support for default system version will not be limited by time.
  • Our system tools will be able to switch to Python 3, drop the burden of Python 2 support and use new features of Python 3.
  • As a distribution that stays close to upstream, Fedora should help Python community go forward by contributing patches and working closely with upstreams to get this accomplished. Thus this Change is meant to benefit not only Fedora, but also broader Python community.
  • Switching to Python 3 as a default will once again push Fedora to stay as close to upstream as possible, highlighting the "Features" and "First" (although, to be honest, Arch Linux was first in this...)

Scope

The main goal is switching to Python 3 as a default, in which state:

  • DNF is the default package manager instead of Yum, which only works with Python 2
  • Python 3 is the only Python implementation in the minimal buildroot
  • Python 3 is the only Python implementation on the LiveCD
  • Anaconda and all of its dependencies run on Python 3
  • cloud-init and all of its dependencies run on Python 3

(see Dependencies for the list of packages that need to be ported)

This will also require revisiting Python guidelines (broader discussion with community and FPC approval - TBD). The result of the discussion will reflect in this Change in further instructions for Fedora packagers.

There are basically two types of packages that need to undergo the conversion:

  • Python extension modules and libraries that provide Python bindings - assuming that there is upstream support, these can receive python3- subpackage anytime without any damage to Fedora; we can then just utilize this subpackage when switching to Python 3 (instead of using current python- subpackage).
  • Packages that build with some sort of "embedded Python support", like gdb, or Rhythmbox with its plugins. In these cases, it makes no sense to do a python3- subpackage, since the whole package would need to be duplicated (e.g. python3-gdb). These packages should be built with Python 3 in Fedora 22 rawhide as soon as possible.

Work in Fedora 21 Timeframe

  • Proposal owners:
    • Discussing changes in Python packaging guidelines with Fedora community and FPC
    • Helping upstreams with porting to Python 3
    • Introducing python3- packages where appropriate, testing packages that only build with Python once (e.g. gdb, Rhythmbox)
  • Other developers:
    • Hopefully the same as proposal owners.
  • Release engineering:
    • Nothing.
  • Policies and guidelines:
    • As mentioned above, this will require a discussion with community and FPC and preparation of changes to Python packaging guidelines. The changes related to the actual switch should however be merged in F22 timeframe and only if the switch is successful.

Work in Fedora 22 Timeframe

  • Proposal owners:
    • Continue the work from F21 timeframe
    • Modify comps accordingly
    • Apply the changes to Python packaging guidelines
  • Other developers:
    • Introduce python3- subpackages where appropriate, build against Python 3 if the package supports it
  • Release engineering:
    • Nothing
  • Policies and guidelines:
    • Apply the changes to Python guidelines if the switch is achieved

Upgrade/compatibility impact

The Python 2 stack will stay in Fedora, it will just not be the default one. Depending on the modifications done to Python packaging guidelines, this probably means that Python 2 packages will stay the way they are and default tools will drag in python3- dependencies. Upstream recommends that /usr/bin/python point to Python 2 runtime for the time being, so if we go with that, there shouldn't be any serious compatibility impact:

  • Users will still be able to install Python 2 packages
  • Both Python 2 and 3 stacks will still live in parallel

Libraries that are built with Python only once (like gdb) may force users to rewrite their custom scripts and plugins (see e.g. https://bugzilla.redhat.com/show_bug.cgi?id=1014549#c3).

How To Test

  • No special hardware is needed.
  • Install Fedora 22 in mock/virtual machine and test if everything still works, most importantly system tools like DNF or Firewalld.
  • Test installing the system with Anaconda running on Python 3.

User Experience

Users shouldn't notice any changes, except that packages in minimal buildroot and on LiveCD will be python3-, not python-. Anaconda deps will be python3- as well. /usr/bin/python will still point to Python 2 and depending on result of discussions with FPC, "yum install python-foo" will still install Python 2 version of the package.

Dependencies

See https://fedoraproject.org/wiki/User:Churchyard/python3 (our tracking page with notes) or https://bugzilla.redhat.com/show_bug.cgi?id=1014209 (tracking bug).

Contingency Plan

  • Contingency mechanism: None needed. Packages that will be ready will be built with Python 3, the rest will be ported in next release.
  • Contingency deadline: Software string freeze
  • Blocks release? No

Documentation

http://docs.python.org/dev/howto/pyporting.html

http://docs.python.org/3/howto/cporting.html

https://wiki.gnome.org/PyGObject/IntrospectionPorting

Release Notes

Retrieved from "https://fedoraproject.org/w/index.php?title=Changes/Python_3_as_Default&oldid=375580"