Churchyard (talk | contribs) |
Churchyard (talk | contribs) |
||
(30 intermediate revisions by 6 users not shown) | |||
Line 22: | Line 22: | ||
<!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name. This keeps all change proposals in the same namespace --> | <!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name. This keeps all change proposals in the same namespace --> | ||
= Platform Python Stack <!-- The name of your change proposal --> = | = Platform Python Stack <!-- The name of your change proposal --> = | ||
{{admon/important|Reverted|This change was reverted, because it was a bad idea entirely. See RHBZ#1483342 and the link below this box.}} | |||
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/SEFUWW4XZBTVOAQ36XOJQ72PIICMFOSN/ | |||
== Summary == | == Summary == | ||
Line 28: | Line 31: | ||
A revisit of the [[Changes/System Python|System Python]] change from [[Releases/24 | Fedora 24]]. It has been renamed from '''System Python''' to '''Platform Python''' not to collide with upstream [https://www.python.org/dev/peps/pep-0432/#a-system-python-executable PEP 432]. | A revisit of the [[Changes/System Python|System Python]] change from [[Releases/24 | Fedora 24]]. It has been renamed from '''System Python''' to '''Platform Python''' not to collide with upstream [https://www.python.org/dev/peps/pep-0432/#a-system-python-executable PEP 432]. | ||
Platform Python will be a separate stack fo Python packages | Platform Python will be a separate stack fo Python packages aimed to provide all necessary dependencies to run and build dnf. In [[Releases/27 | Fedora 27]] this will be Python 3.6 (same version as {{package|python3}}, but this might change in the future with the Modularity effort). | ||
== Owner == | == Owner == | ||
Line 36: | Line 39: | ||
--> | --> | ||
* Name: [[User:Churchyard|Miro Hrončok]] | * Name: [[User:Churchyard|Miro Hrončok]] | ||
* Name: [[User:Cstratak|Charalampos Stratakis]] | |||
* Name: [[User:Torsava|Tomáš Orsava]] | |||
<!-- Include you email address that you can be reached should people want to contact you about helping with your change, status is requested, or technical issues need to be resolved. If the change proposal is owned by a SIG, please also add a primary contact person. --> | <!-- Include you email address that you can be reached should people want to contact you about helping with your change, status is requested, or technical issues need to be resolved. If the change proposal is owned by a SIG, please also add a primary contact person. --> | ||
* Email: <python-maint at redhat.com> | * Email: <python-maint at redhat.com> | ||
Line 58: | Line 63: | ||
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development | CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development | ||
--> | --> | ||
* Tracker bug: | * Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1479272 #1479272] | ||
== Detailed Description == | == Detailed Description == | ||
Line 66: | Line 71: | ||
The changes made during the [[Changes/System Python|System Python]] change from [[Releases/24 | Fedora 24]] will be reverted. | The changes made during the [[Changes/System Python|System Python]] change from [[Releases/24 | Fedora 24]] will be reverted. | ||
A new platform-python (source) package will be created as a copy of the {{package|python3}} package | A new platform-python (source) package will be created as a copy of the {{package|python3}} package. Unlike python3, it will be changed so that: | ||
* the location of files will not be <code>/usr/lib/python3.6/</code> but <code>/usr/lib/platform- | * the location of files will not be <code>/usr/lib/python3.6/</code> but <code>/usr/lib/platform-python3.6/</code> | ||
** the script that generates the <code>python(abi)</code> requirement will be adapted to generate <code>platform-python(abi)</code> requirement for that location | ** the script that generates the <code>python(abi)</code> requirement will be adapted to generate <code>platform-python(abi)</code> requirement for that location | ||
* the location of the binary will be <code>/usr/libexec/platform-python</code> | * the location of the binary will be <code>/usr/libexec/platform-python</code> | ||
* all the macros will use the platform python binary and will be renamed | |||
* all subpackages will be named <code>platform-python-*</code> | * all subpackages will be named <code>platform-python-*</code> | ||
* the debug build will be disabled | * the debug build will be disabled | ||
* the modules that are (before this change) not part of <code>system-python-libs</code> but rather <code>python3-libs</code> will relocate to <code>platform-python-libs-devel</code> to be available only on build-time | * the modules that are (before this change) not part of <code>system-python-libs</code> but rather <code>python3-libs</code> will relocate to <code>platform-python-libs-devel</code> to be available only on build-time | ||
* <code>platform-python | * <code>platform-python</code> will provide <code>platform-python(abi)</code> instead of <code>python(abi)</code> | ||
Several packages (see bellow) will | Several packages (see bellow) will introduce a new <code>platform-python-</code> subpackage. Example specfile of such package: | ||
%global pypi_name foo | %global pypi_name foo | ||
%bcond_without python2 | |||
%bcond_without python3 | |||
%bcond_without platform_python | |||
Name: python-%{pypi_name} | Name: python-%{pypi_name} | ||
Line 89: | Line 99: | ||
Source0: https://files.pythonhosted.org/packages/source/f/%{pypi_name}/%{pypi_name}-%{version}.tar.gz | Source0: https://files.pythonhosted.org/packages/source/f/%{pypi_name}/%{pypi_name}-%{version}.tar.gz | ||
%if %{with python2} | |||
%if | |||
BuildRequires: python2-devel | BuildRequires: python2-devel | ||
BuildRequires: python2-setuptools | BuildRequires: python2-setuptools | ||
%endif | |||
BuildRequires: | %if %{with python3} | ||
BuildRequires: | BuildRequires: python3-devel | ||
BuildRequires: python3-setuptools | |||
%endif | |||
%if %{with platform_python} | |||
BuildRequires: platform-python-devel | |||
BuildRequires: platform-python-setuptools | |||
%endif | |||
BuildArch: noarch | BuildArch: noarch | ||
Line 109: | Line 119: | ||
... | ... | ||
%if %{with python2} | |||
%package -n python2-%{pypi_name} | %package -n python2-%{pypi_name} | ||
Summary: %{summary} | Summary: %{summary} | ||
Line 116: | Line 127: | ||
%description -n python2-%{pypi_name} | %description -n python2-%{pypi_name} | ||
... | ... | ||
%endif | |||
%package -n | %if %{with python3} | ||
%package -n python3-%{pypi_name} | |||
Summary: %{summary} | Summary: %{summary} | ||
Requires: | Requires: python3-bar | ||
%{?python_provide:%python_provide | %{?python_provide:%python_provide python3-%{pypi_name}} | ||
%description -n | %description -n python3-%{pypi_name} | ||
... | ... | ||
%endif | |||
%if %{with platform_python} | |||
%package -n platform-python-%{pypi_name} | |||
Summary: %{summary} | |||
Requires: platform-python-bar | |||
%description -n platform-python-%{pypi_name} | |||
... | |||
%endif | |||
Line 132: | Line 155: | ||
%build | %build | ||
%if %{with python2} | |||
%py2_build | %py2_build | ||
%endif | |||
%if %{with python3} | |||
%py3_build | %py3_build | ||
%endif | |||
%if %{with platform_python} | |||
%platform_py_build | |||
%endif | |||
%install | %install | ||
%if %{with python2} | |||
%py2_install | %py2_install | ||
rm %{buildroot}%{_bindir}/%{pypi_name} | rm %{buildroot}%{_bindir}/%{pypi_name} | ||
%endif | |||
%if %{with platform_python} | |||
%platform_py_install | |||
rm %{buildroot}%{_bindir}/%{pypi_name} | |||
%endif | |||
%if %{with python3} | |||
%py3_install | %py3_install | ||
%endif | |||
%check | %check | ||
%if %{with python2} | |||
export PYTHONPATH=%{buildroot}%{python2_sitelib} | export PYTHONPATH=%{buildroot}%{python2_sitelib} | ||
%{__python2} -m unittest discover -v | %{__python2} -m unittest discover -v | ||
%endif | |||
%if %{with python3} | |||
export PYTHONPATH=%{buildroot}%{python3_sitelib} | export PYTHONPATH=%{buildroot}%{python3_sitelib} | ||
%{__python3} -m unittest discover -v | %{__python3} -m unittest discover -v | ||
%endif | |||
%if %{with platform_python} | |||
export PYTHONPATH=%{buildroot}%{platform_python_sitelib} | |||
%{__platform_python} -m unittest discover -v | |||
%endif | |||
%if %{with python2} | |||
%files -n python2-%{pypi_name} | %files -n python2-%{pypi_name} | ||
%doc README.rst CHANGES.rst | %doc README.rst CHANGES.rst | ||
Line 155: | Line 207: | ||
%{python2_sitelib}/%{pypi_name}.so | %{python2_sitelib}/%{pypi_name}.so | ||
%{python2_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info | %{python2_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info | ||
%endif | |||
%files -n | %if %{with python3} | ||
%files -n python3-%{pypi_name} | |||
%doc README.rst CHANGES.rst | %doc README.rst CHANGES.rst | ||
%license COPYING | %license COPYING | ||
%{_bindir}/%{pypi_name} | |||
%{python3_sitelib}/%{pypi_name}.cpython-%{python3_version_nodots}*.so | %{python3_sitelib}/%{pypi_name}.cpython-%{python3_version_nodots}*.so | ||
%{python3_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info | %{python3_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info | ||
%endif | |||
%if %{with platform_python} | |||
%files -n platform-python-%{pypi_name} | |||
%doc README.rst CHANGES.rst | |||
%license COPYING | |||
%{platform_python_sitelib}/%{pypi_name}.cpython-%{platform_python_version_nodots}*.so | |||
%{platform_python_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info | |||
%endif | |||
%changelog | %changelog | ||
... | ... | ||
The complete list of affected packages is not final yet. It should be all the Python build dependencies of dnf and their build dependencies as well, excluding packages only needed to build the documentation ( | The complete list of affected packages is not final yet. It should be all the Python build dependencies of dnf and their build dependencies as well, excluding packages only needed to build the documentation (normal python3- (or even python2-) packages will be used to build the documentation as it is now). | ||
Currently, we have established this as a list of packages: | Currently, we have established this as a list of packages: | ||
dnf | |||
gpgme | gpgme | ||
libcomps | libcomps | ||
Line 183: | Line 239: | ||
librepo | librepo | ||
pytest | pytest | ||
python-coverage | python-coverage | ||
python-hypothesis | python-hypothesis | ||
python-iniparse | python-iniparse | ||
python-nose | python-nose | ||
python-py | python-py | ||
python- | python-rpm-generators | ||
python-rpm-macros | python-rpm-macros | ||
python-setuptools | python-setuptools | ||
python-setuptools_scm | python-setuptools_scm | ||
python-six | python-six | ||
rpm | rpm | ||
Only the following list will be installable from the platform module (the rest is merely meant as a build dependency): | Only the following list will be installable from the platform module (the rest is merely meant as a build dependency): | ||
platform-python | platform-python | ||
platform-python- | platform-python-iniparse | ||
platform-python-gpg | |||
platform-python-rpm | |||
platform-python-dnf | platform-python-dnf | ||
platform-python- | platform-python-libs | ||
platform-python- | platform-python-six | ||
platform-python-libcomps | platform-python-libcomps | ||
platform-python-librepo | platform-python-librepo | ||
platform-python- | platform-python-hawkey | ||
(And other subpackages from their source RPMs needed as dependencies for those, such as gpgme etc.) | |||
== Benefit to Fedora == | == Benefit to Fedora == | ||
With the Modularity effort a minimal stack of Python only needed to run dnf is needed to be part of the Platform. Changing System Python to be a separate stack will allow us to do that. | With the Modularity effort a minimal stack of Python only needed to run dnf (and potentially other system management utilities later on) is needed to be part of the Platform. Changing System/Platform Python to be a separate stack will allow us to do that. In the future that allows us to have different version of Python in Platform and in the "Python 3" module. | ||
Splitting the full Python stack away from Platform allows for flexible and fast paced, upstream-driven delivery, independent of the main Fedora release cycle. See [https://docs.pagure.org/modularity Modularity] for more information about this concept in general and the [[Changes/Host_and_Platform|Host and Platform change proposal]] to learn more about the abovementioned Platform module. | |||
<!-- What is the benefit to the platform? If this is a major capability update, what has changed? If this is a new functionality, what capabilities does it bring? Why will Fedora become a better distribution or project because of this proposal?--> | <!-- What is the benefit to the platform? If this is a major capability update, what has changed? If this is a new functionality, what capabilities does it bring? Why will Fedora become a better distribution or project because of this proposal?--> | ||
Line 253: | Line 277: | ||
== Scope == | == Scope == | ||
* Proposal owners: | * Proposal owners: | ||
** A [[Changes/Platform_Python_Stack/Roadmap|Roadmap]] has been created | |||
** identify a list of runtime and build dependencies of dnf | ** identify a list of runtime and build dependencies of dnf | ||
** create a <code> | ** create a <code>platform-python</code> source package | ||
** change the script that generates the <code>python(abi)</code> requirements to also generate <code> | ** change the script that generates the <code>python(abi)</code> requirements to also generate <code>platform-python(abi)</code> requirements | ||
** revert system-python related things from the {{package|python3}} package | ** revert system-python related things from the {{package|python3}} package (add appropriate provides/obsoletes) | ||
** add a <code> | ** add a <code>platform-python-</code> subpackage to identified list of packages | ||
** switch dnf to use that stack | ** switch dnf to use that stack | ||
<!-- What work do the feature owners have to accomplish to complete the feature in time for release? Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?--> | <!-- What work do the feature owners have to accomplish to complete the feature in time for release? Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?--> | ||
Line 264: | Line 289: | ||
<!-- What work do other developers have to accomplish to complete the feature in time for release? Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?--> | <!-- What work do other developers have to accomplish to complete the feature in time for release? Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?--> | ||
* Release engineering: [https://pagure.io/releng/ | * Release engineering: [https://pagure.io/releng/issue/6917 #6917] (a check of an impact with Release Engineering is needed) <!-- REQUIRED FOR SYSTEM WIDE AS WELL AS FOR SELF CONTAINED CHANGES --> | ||
<!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)? Is a mass rebuild required? include a link to the releng issue. | <!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)? Is a mass rebuild required? include a link to the releng issue. | ||
The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing, and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication --> | The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing, and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication --> | ||
Line 324: | Line 349: | ||
If not done on time: Keep dnf running on {{package|python3}}. Finish the work in time of Fedora 28. | If not done on time: Keep dnf running on {{package|python3}}. Finish the work in time of Fedora 28. | ||
If proven not to work as designed: Retire | If proven not to work as designed: Retire platform-python package and remove all the platform-python- subpackages. Provide/obsolete platform-python- subpackages from their python3- counterparts. | ||
== Documentation == | == Documentation == | ||
Line 339: | Line 364: | ||
--> | --> | ||
[[Category: | [[Category:ChangeAcceptedF27]] | ||
<!-- When your change proposal page is completed and ready for review and announcement --> | <!-- When your change proposal page is completed and ready for review and announcement --> | ||
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler --> | <!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler --> |
Latest revision as of 20:34, 25 August 2020
Platform Python Stack
Summary
A revisit of the System Python change from Fedora 24. It has been renamed from System Python to Platform Python not to collide with upstream PEP 432.
Platform Python will be a separate stack fo Python packages aimed to provide all necessary dependencies to run and build dnf. In Fedora 27 this will be Python 3.6 (same version as
, but this might change in the future with the Modularity effort).
python3
Owner
- Name: Miro Hrončok
- Name: Charalampos Stratakis
- Name: Tomáš Orsava
- Email: <python-maint at redhat.com>
- Release notes owner:
Current status
Detailed Description
The changes made during the System Python change from Fedora 24 will be reverted.
A new platform-python (source) package will be created as a copy of the
package. Unlike python3, it will be changed so that:
python3
- the location of files will not be
/usr/lib/python3.6/
but/usr/lib/platform-python3.6/
- the script that generates the
python(abi)
requirement will be adapted to generateplatform-python(abi)
requirement for that location
- the script that generates the
- the location of the binary will be
/usr/libexec/platform-python
- all the macros will use the platform python binary and will be renamed
- all subpackages will be named
platform-python-*
- the debug build will be disabled
- the modules that are (before this change) not part of
system-python-libs
but ratherpython3-libs
will relocate toplatform-python-libs-devel
to be available only on build-time platform-python
will provideplatform-python(abi)
instead ofpython(abi)
Several packages (see bellow) will introduce a new platform-python-
subpackage. Example specfile of such package:
%global pypi_name foo %bcond_without python2 %bcond_without python3 %bcond_without platform_python Name: python-%{pypi_name} Version: 0.13 Release: 1%{?dist} Summary: Python %{pypi_name} module License: GPLv2 URL: https://github.com/fedora-python/%{pypi_name} Source0: https://files.pythonhosted.org/packages/source/f/%{pypi_name}/%{pypi_name}-%{version}.tar.gz %if %{with python2} BuildRequires: python2-devel BuildRequires: python2-setuptools %endif %if %{with python3} BuildRequires: python3-devel BuildRequires: python3-setuptools %endif %if %{with platform_python} BuildRequires: platform-python-devel BuildRequires: platform-python-setuptools %endif BuildArch: noarch %description ... %if %{with python2} %package -n python2-%{pypi_name} Summary: %{summary} Requires: python2-bar %{?python_provide:%python_provide python2-%{pypi_name}} %description -n python2-%{pypi_name} ... %endif %if %{with python3} %package -n python3-%{pypi_name} Summary: %{summary} Requires: python3-bar %{?python_provide:%python_provide python3-%{pypi_name}} %description -n python3-%{pypi_name} ... %endif %if %{with platform_python} %package -n platform-python-%{pypi_name} Summary: %{summary} Requires: platform-python-bar %description -n platform-python-%{pypi_name} ... %endif %prep %autosetup -n %{pypi_name}-%{version} %build %if %{with python2} %py2_build %endif %if %{with python3} %py3_build %endif %if %{with platform_python} %platform_py_build %endif %install %if %{with python2} %py2_install rm %{buildroot}%{_bindir}/%{pypi_name} %endif %if %{with platform_python} %platform_py_install rm %{buildroot}%{_bindir}/%{pypi_name} %endif %if %{with python3} %py3_install %endif %check %if %{with python2} export PYTHONPATH=%{buildroot}%{python2_sitelib} %{__python2} -m unittest discover -v %endif %if %{with python3} export PYTHONPATH=%{buildroot}%{python3_sitelib} %{__python3} -m unittest discover -v %endif %if %{with platform_python} export PYTHONPATH=%{buildroot}%{platform_python_sitelib} %{__platform_python} -m unittest discover -v %endif %if %{with python2} %files -n python2-%{pypi_name} %doc README.rst CHANGES.rst %license COPYING %{python2_sitelib}/%{pypi_name}.so %{python2_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info %endif %if %{with python3} %files -n python3-%{pypi_name} %doc README.rst CHANGES.rst %license COPYING %{_bindir}/%{pypi_name} %{python3_sitelib}/%{pypi_name}.cpython-%{python3_version_nodots}*.so %{python3_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info %endif %if %{with platform_python} %files -n platform-python-%{pypi_name} %doc README.rst CHANGES.rst %license COPYING %{platform_python_sitelib}/%{pypi_name}.cpython-%{platform_python_version_nodots}*.so %{platform_python_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info %endif %changelog ...
The complete list of affected packages is not final yet. It should be all the Python build dependencies of dnf and their build dependencies as well, excluding packages only needed to build the documentation (normal python3- (or even python2-) packages will be used to build the documentation as it is now).
Currently, we have established this as a list of packages:
dnf gpgme libcomps libdnf librepo pytest python-coverage python-hypothesis python-iniparse python-nose python-py python-rpm-generators python-rpm-macros python-setuptools python-setuptools_scm python-six rpm
Only the following list will be installable from the platform module (the rest is merely meant as a build dependency):
platform-python platform-python-iniparse platform-python-gpg platform-python-rpm platform-python-dnf platform-python-libs platform-python-six platform-python-libcomps platform-python-librepo platform-python-hawkey
(And other subpackages from their source RPMs needed as dependencies for those, such as gpgme etc.)
Benefit to Fedora
With the Modularity effort a minimal stack of Python only needed to run dnf (and potentially other system management utilities later on) is needed to be part of the Platform. Changing System/Platform Python to be a separate stack will allow us to do that. In the future that allows us to have different version of Python in Platform and in the "Python 3" module.
Splitting the full Python stack away from Platform allows for flexible and fast paced, upstream-driven delivery, independent of the main Fedora release cycle. See Modularity for more information about this concept in general and the Host and Platform change proposal to learn more about the abovementioned Platform module.
Scope
- Proposal owners:
- A Roadmap has been created
- identify a list of runtime and build dependencies of dnf
- create a
platform-python
source package - change the script that generates the
python(abi)
requirements to also generateplatform-python(abi)
requirements - revert system-python related things from the
package (add appropriate provides/obsoletes)python3
- add a
platform-python-
subpackage to identified list of packages - switch dnf to use that stack
- Other developers: N/A (not a System Wide Change)
- Release engineering: #6917 (a check of an impact with Release Engineering is needed)
- List of deliverables: N/A (not a System Wide Change)
- Policies and guidelines: N/A (not a System Wide Change)
- Trademark approval: N/A (not needed for this Change)
Upgrade/compatibility impact
N/A (not a System Wide Change)
How To Test
N/A (not a System Wide Change)
User Experience
Nothing should change for the user, except a slightly larger amount of packages will be installed on the system.
Dependencies
N/A (not a System Wide Change)
Contingency Plan
- Contingency mechanism: (What to do? Who will do it?) N/A (not a System Wide Change)
- Contingency deadline: N/A (not a System Wide Change)
- Blocks release? No (not a System Wide Change)
- Blocks product? No
If not done on time: Keep dnf running on
. Finish the work in time of Fedora 28.
python3
If proven not to work as designed: Retire platform-python package and remove all the platform-python- subpackages. Provide/obsolete platform-python- subpackages from their python3- counterparts.
Documentation
N/A (not a System Wide Change)