Mark Python base environments as “externally managed”
PEP 668 – Marking Python base environments as “externally managed” proposes a mechanism for a Python installation to communicate to tools like pip that its global package installation context is managed by some means external to Python, such as an OS package manager. We will mark our Python installation in
/usr/lib(64)/python3.X/ as managed by rpm.
- Name: Miro Hrončok
- Email: email@example.com
- Targeted release: Fedora Linux 38
- Last updated: 2022-10-07
- FESCo issue: <will be assigned by the Wrangler>
- Tracker bug: <will be assigned by the Wrangler>
- Release notes tracker: <will be assigned by the Wrangler>
See PEP 668 – Marking Python base environments as “externally managed”.
We'll add the PEP-described marker file to the
python3-libs package. The file will be installed at
/usr/lib64/python3.11/EXTERNALLY-MANAGED on a 64bt architecture and Python 3.11.
The file will contain:
[externally-managed] Error=To install Python packages system-wide, try
dnf install python3-xyz, where xyz is the package you are trying to install. If you wish to install a non-RPM-packaged Python package, create a virtual environment using
python3 -m venv path/to/venv. Then use path/to/venv/bin/python and path/to/venv/bin/pip. If you wish to install a non-RPM-packaged Python application, it may be easiest to use
pipx install xyz, which will manage a virtual environment for you. Make sure you have pipx installed via dnf.
Following the PEPs distro recommendations, we will have 2 installation schemes. One for RPM packages and one for pip (and similar tools). Packages installed via pip will continue to be installed to
/usr/local/lib(64) by default. SUers attempting to install via pip to
/usr/lib(64) will see the above error unless they suppress it (the exact way of doing that is still to be determined).
Unfortunately, as of writing this change proposal, pip does not yet respect this marker file. The upstream work is tracked at https://github.com/pypa/pip/issues/11381 and the change owners plan to coordinate with pip upstream on this. Whether or not we will keep the marker even if pip still won't support PEP 668 will be decided before the contingency deadline.
Benefit to Fedora
Users will be less likely to brick their system with
pip and this protection will be more aligned with the upstream recommendations.
- Proposal owners:
- work with pip upstream to make it support the PEP
- add the marker as described in the PEP
- split the current sysconfig installation scheme into two and make sure the marker only affects one of them
- Other developers: N/A (not needed for this Change)
- Release engineering: N/A (not needed for this Change)
- Policies and guidelines: N/A (not needed for this Change)
- Trademark approval: N/A (not needed for this Change)
- Alignment with Objectives: N/A (not needed for this Change)
How To Test
- Contingency mechanism: revert the change
- Contingency deadline: final freeze
- Blocks release? No
N/A (not a System Wide Change)