From Fedora Project Wiki
(→‎Detailed Description: Add bcond's for python2, python3, platform_python)
 
(8 intermediate revisions by 3 users not shown)
Line 22: Line 22:
 
<!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->
 
<!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->
 
= Platform Python Stack <!-- The name of your change proposal --> =
 
= Platform Python Stack <!-- The name of your change proposal --> =
 +
{{admon/important|Reverted|This change was reverted, because it was a bad idea entirely. See RHBZ#1483342 and the link below this box.}}
 +
 +
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/SEFUWW4XZBTVOAQ36XOJQ72PIICMFOSN/
  
 
== Summary ==
 
== Summary ==
Line 37: Line 40:
 
* Name: [[User:Churchyard|Miro Hrončok]]
 
* Name: [[User:Churchyard|Miro Hrončok]]
 
* Name: [[User:Cstratak|Charalampos Stratakis]]
 
* Name: [[User:Cstratak|Charalampos Stratakis]]
 +
* Name: [[User:Torsava|Tomáš Orsava]]
 
<!-- Include you email address that you can be reached should people want to contact you about helping with your change, status is requested, or technical issues need to be resolved. If the change proposal is owned by a SIG, please also add a primary contact person. -->
 
<!-- Include you email address that you can be reached should people want to contact you about helping with your change, status is requested, or technical issues need to be resolved. If the change proposal is owned by a SIG, please also add a primary contact person. -->
 
* Email: <python-maint at redhat.com>
 
* Email: <python-maint at redhat.com>
Line 59: Line 63:
 
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
 
CLOSED as NEXTRELEASE -> change is completed and verified and will be delivered in next release under development
 
-->
 
-->
* Tracker bug: <will be assigned by the Wrangler>
+
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1479272 #1479272]
  
 
== Detailed Description ==
 
== Detailed Description ==
Line 160: Line 164:
 
   
 
   
 
  %if %{with platform_python}
 
  %if %{with platform_python}
  %plpy_build
+
  %platform_py_build
 
  %endif
 
  %endif
 
   
 
   
Line 171: Line 175:
 
   
 
   
 
  %if %{with platform_python}
 
  %if %{with platform_python}
  %plpy_install
+
  %platform_py_install
 
  rm %{buildroot}%{_bindir}/%{pypi_name}
 
  rm %{buildroot}%{_bindir}/%{pypi_name}
 
  %endif
 
  %endif
Line 192: Line 196:
 
   
 
   
 
  %if %{with platform_python}
 
  %if %{with platform_python}
  export PYTHONPATH=%{buildroot}%{platformpython_sitelib}
+
  export PYTHONPATH=%{buildroot}%{platform_python_sitelib}
  %{__platformpython} -m unittest discover -v
+
  %{__platform_python} -m unittest discover -v
 
  %endif
 
  %endif
 
   
 
   
Line 218: Line 222:
 
  %doc README.rst CHANGES.rst
 
  %doc README.rst CHANGES.rst
 
  %license COPYING
 
  %license COPYING
  %{platformpython_sitelib}/%{pypi_name}.cpython-%{platformplatformpython_version_nodots}*.so
+
  %{platform_python_sitelib}/%{pypi_name}.cpython-%{platform_python_version_nodots}*.so
  %{platformpython_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info
+
  %{platform_python_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info
 
  %endif
 
  %endif
 
   
 
   
Line 229: Line 233:
 
Currently, we have established this as a list of packages:
 
Currently, we have established this as a list of packages:
  
  dbus-python
+
  dnf
 
  gpgme
 
  gpgme
 
  libcomps
 
  libcomps
Line 235: Line 239:
 
  librepo
 
  librepo
 
  pytest
 
  pytest
python-argcomplete
 
python-cffi
 
python-contextlib2
 
 
  python-coverage
 
  python-coverage
python-cryptography
 
python-decorator
 
python-extras
 
python-fixtures
 
python-funcsigs
 
 
  python-hypothesis
 
  python-hypothesis
python-idna
 
 
  python-iniparse
 
  python-iniparse
python-jinja2
 
python-jsonschema
 
python-keyring
 
python-linecache2
 
python-markupsafe
 
python-mimeparse
 
python-mock
 
 
  python-nose
 
  python-nose
python-pbr
 
python-pexpect
 
python-pip
 
python-ply
 
python-ptyprocess
 
 
  python-py
 
  python-py
  python-pyasn1
+
  python-rpm-generators
python-pycparser
 
python-pygpgme
 
python-pytest-timeout
 
 
  python-rpm-macros
 
  python-rpm-macros
python-SecretStorage
 
 
  python-setuptools
 
  python-setuptools
 
  python-setuptools_scm
 
  python-setuptools_scm
 
  python-six
 
  python-six
python-testtools
 
python-traceback2
 
python-unittest2
 
python-wheel
 
pyxattr
 
pyxdg
 
 
  rpm
 
  rpm
 
 
 
 
Only the following list will be installable from the platform module (the rest is merely meant as a build dependency):
 
Only the following list will be installable from the platform module (the rest is merely meant as a build dependency):
  
 
  platform-python
 
  platform-python
 +
platform-python-iniparse
 +
platform-python-gpg
 +
platform-python-rpm
 
  platform-python-dnf
 
  platform-python-dnf
  platform-python-gpg
+
  platform-python-libs
  platform-python-hawkey
+
  platform-python-six
platform-python-iniparse
 
 
  platform-python-libcomps
 
  platform-python-libcomps
 
  platform-python-librepo
 
  platform-python-librepo
  platform-python-libs
+
  platform-python-hawkey
platform-python-pyparsing
 
platform-python-rpm
 
platform-python-six
 
  
 
(And other subpackages from their source RPMs needed as dependencies for those, such as gpgme etc.)
 
(And other subpackages from their source RPMs needed as dependencies for those, such as gpgme etc.)
Line 393: Line 364:
 
-->
 
-->
  
[[Category:ChangeReadyForFesco]]
+
[[Category:ChangeAcceptedF27]]
 
<!-- When your change proposal page is completed and ready for review and announcement -->
 
<!-- When your change proposal page is completed and ready for review and announcement -->
 
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
 
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->

Latest revision as of 20:34, 25 August 2020


Platform Python Stack

Important.png
Reverted
This change was reverted, because it was a bad idea entirely. See RHBZ#1483342 and the link below this box.

https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/SEFUWW4XZBTVOAQ36XOJQ72PIICMFOSN/

Summary

A revisit of the System Python change from Fedora 24. It has been renamed from System Python to Platform Python not to collide with upstream PEP 432.

Platform Python will be a separate stack fo Python packages aimed to provide all necessary dependencies to run and build dnf. In Fedora 27 this will be Python 3.6 (same version as Package-x-generic-16.pngpython3, but this might change in the future with the Modularity effort).

Owner

Current status

Detailed Description

The changes made during the System Python change from Fedora 24 will be reverted.

A new platform-python (source) package will be created as a copy of the Package-x-generic-16.pngpython3 package. Unlike python3, it will be changed so that:

  • the location of files will not be /usr/lib/python3.6/ but /usr/lib/platform-python3.6/
    • the script that generates the python(abi) requirement will be adapted to generate platform-python(abi) requirement for that location
  • the location of the binary will be /usr/libexec/platform-python
  • all the macros will use the platform python binary and will be renamed
  • all subpackages will be named platform-python-*
  • the debug build will be disabled
  • the modules that are (before this change) not part of system-python-libs but rather python3-libs will relocate to platform-python-libs-devel to be available only on build-time
  • platform-python will provide platform-python(abi) instead of python(abi)

Several packages (see bellow) will introduce a new platform-python- subpackage. Example specfile of such package:

%global pypi_name foo

%bcond_without python2
%bcond_without python3
%bcond_without platform_python

Name:           python-%{pypi_name}
Version:        0.13
Release:        1%{?dist}
Summary:        Python %{pypi_name} module

License:        GPLv2
URL:            https://github.com/fedora-python/%{pypi_name}
Source0:        https://files.pythonhosted.org/packages/source/f/%{pypi_name}/%{pypi_name}-%{version}.tar.gz

%if %{with python2}
BuildRequires:  python2-devel
BuildRequires:  python2-setuptools
%endif

%if %{with python3}
BuildRequires:  python3-devel
BuildRequires:  python3-setuptools
%endif

%if %{with platform_python}
BuildRequires:  platform-python-devel
BuildRequires:  platform-python-setuptools
%endif

BuildArch:      noarch

%description
...

%if %{with python2}
%package -n     python2-%{pypi_name}
Summary:        %{summary}
Requires:       python2-bar
%{?python_provide:%python_provide python2-%{pypi_name}}

%description -n python2-%{pypi_name}
...
%endif

%if %{with python3}
%package -n     python3-%{pypi_name}
Summary:        %{summary}
Requires:       python3-bar
%{?python_provide:%python_provide python3-%{pypi_name}}

%description -n python3-%{pypi_name}
...
%endif


%if %{with platform_python}
%package -n     platform-python-%{pypi_name}
Summary:        %{summary}
Requires:       platform-python-bar

%description -n platform-python-%{pypi_name}
...
%endif


%prep
%autosetup -n %{pypi_name}-%{version}


%build
%if %{with python2}
%py2_build
%endif

%if %{with python3}
%py3_build
%endif

%if %{with platform_python}
%platform_py_build
%endif


%install
%if %{with python2}
%py2_install
rm %{buildroot}%{_bindir}/%{pypi_name}
%endif

%if %{with platform_python}
%platform_py_install
rm %{buildroot}%{_bindir}/%{pypi_name}
%endif

%if %{with python3}
%py3_install
%endif


%check
%if %{with python2}
export PYTHONPATH=%{buildroot}%{python2_sitelib}
%{__python2} -m unittest discover -v
%endif

%if %{with python3}
export PYTHONPATH=%{buildroot}%{python3_sitelib}
%{__python3} -m unittest discover -v
%endif

%if %{with platform_python}
export PYTHONPATH=%{buildroot}%{platform_python_sitelib}
%{__platform_python} -m unittest discover -v
%endif


%if %{with python2}
%files -n python2-%{pypi_name}
%doc README.rst CHANGES.rst
%license COPYING
%{python2_sitelib}/%{pypi_name}.so
%{python2_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info
%endif

%if %{with python3}
%files -n python3-%{pypi_name}
%doc README.rst CHANGES.rst
%license COPYING
%{_bindir}/%{pypi_name}
%{python3_sitelib}/%{pypi_name}.cpython-%{python3_version_nodots}*.so
%{python3_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info
%endif

%if %{with platform_python}
%files -n platform-python-%{pypi_name}
%doc README.rst CHANGES.rst
%license COPYING
%{platform_python_sitelib}/%{pypi_name}.cpython-%{platform_python_version_nodots}*.so
%{platform_python_sitelib}/%{pypi_name}-%{version}-py?.?.egg-info
%endif

%changelog
...

The complete list of affected packages is not final yet. It should be all the Python build dependencies of dnf and their build dependencies as well, excluding packages only needed to build the documentation (normal python3- (or even python2-) packages will be used to build the documentation as it is now).

Currently, we have established this as a list of packages:

dnf
gpgme
libcomps
libdnf
librepo
pytest
python-coverage
python-hypothesis
python-iniparse
python-nose
python-py
python-rpm-generators
python-rpm-macros
python-setuptools
python-setuptools_scm
python-six
rpm

Only the following list will be installable from the platform module (the rest is merely meant as a build dependency):

platform-python
platform-python-iniparse
platform-python-gpg
platform-python-rpm
platform-python-dnf
platform-python-libs
platform-python-six
platform-python-libcomps
platform-python-librepo
platform-python-hawkey

(And other subpackages from their source RPMs needed as dependencies for those, such as gpgme etc.)

Benefit to Fedora

With the Modularity effort a minimal stack of Python only needed to run dnf (and potentially other system management utilities later on) is needed to be part of the Platform. Changing System/Platform Python to be a separate stack will allow us to do that. In the future that allows us to have different version of Python in Platform and in the "Python 3" module.

Splitting the full Python stack away from Platform allows for flexible and fast paced, upstream-driven delivery, independent of the main Fedora release cycle. See Modularity for more information about this concept in general and the Host and Platform change proposal to learn more about the abovementioned Platform module.


Scope

  • Proposal owners:
    • A Roadmap has been created
    • identify a list of runtime and build dependencies of dnf
    • create a platform-python source package
    • change the script that generates the python(abi) requirements to also generate platform-python(abi) requirements
    • revert system-python related things from the Package-x-generic-16.pngpython3 package (add appropriate provides/obsoletes)
    • add a platform-python- subpackage to identified list of packages
    • switch dnf to use that stack
  • Other developers: N/A (not a System Wide Change)
  • Release engineering: #6917 (a check of an impact with Release Engineering is needed)
  • Policies and guidelines: N/A (not a System Wide Change)
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

N/A (not a System Wide Change)

How To Test

N/A (not a System Wide Change)

User Experience

Nothing should change for the user, except a slightly larger amount of packages will be installed on the system.

Dependencies

N/A (not a System Wide Change)

Contingency Plan

  • Contingency mechanism: (What to do? Who will do it?) N/A (not a System Wide Change)
  • Contingency deadline: N/A (not a System Wide Change)
  • Blocks release? No (not a System Wide Change)
  • Blocks product? No

If not done on time: Keep dnf running on Package-x-generic-16.pngpython3. Finish the work in time of Fedora 28.

If proven not to work as designed: Retire platform-python package and remove all the platform-python- subpackages. Provide/obsolete platform-python- subpackages from their python3- counterparts.

Documentation

N/A (not a System Wide Change)

Release Notes