Changes/Setuptools 74+

Setuptools 74+

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Summary

Update to a new upstream release of python-setuptools that is not completely compatible with previous releases. Most notably, version 72.0.0 removed support for the setup.py test command (deprecated for 5 years). This is a breaking change and Fedora packages that use the setup.py test command during the build need to be adapted to use a different test runner, such as unittest, pytest, etc. There might be other breaking changes. We estimate 142 to 196 Fedora Rawhide packages to fail to build due to this upgrade (as of Sep 2024).

Owner

• Name: Miro Hrončok
• Email: mhroncok@redhat.com

Current status

• Targeted release: Fedora Linux 42
• Last updated: 2024-09-24
• Announced
• Discussion thread
• FESCo issue: <will be assigned by the Wrangler>
• Tracker bug: <will be assigned by the Wrangler>
• Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

Setuptools is a package development process library designed to facilitate packaging Python projects by enhancing the former Python standard library distutils (distribution utilities). It changes fast and it often introduces breaking changes such as the removal of the setup.py test command in version 72.0.0 which was deprecated since 2019.

If your Fedora package still uses the setup.py test command (deprecated for 5 years), please update to %pytest, %tox, %{python3} -m unittest, etc.

List of packages that will likely fail to build, because they use the removed setup.py test command in the specfile:

$ rg '^[^-#].+(\b|_)setup(\.py|\})\s+test\b' -l | sed -E 's|^rpm-specs/(.+)\.spec$|\1|' | sort
adapt
ansible-inventory-grapher
binwalk
btrfs-sxbackup
calibre
cepces
classification-banner
ddiskit
fprettify
fros
grammalecte
gtts
gtts-token
hddfancontrol
hgsvn
ioc-writer
isrcsubmit
kitty
lazygal
libvirt-test-API
mopidy
mopidy-mpd
nototools
past-time
petsc
pycairo
PyGreSQL
pyjokes
pyowm
python3-lxc
python3-mallard-ducktype
python-aioresponses
python-apsw
python-auth-credential
python-autopage
python-binaryornot
python-booleanoperations
python-bracex
python-can
python-case
python-casttube
python-cccolutils
python-chai
python-cliff
python-collada
python-colorclass
python-CommonMark
python-cpuinfo
python-cwcwidth
python-cx-oracle
python-decorator
python-dfdatetime
python-dictdiffer
python-digitalocean
python-dirq
python-distroinfo
python-django-contrib-comments
python-django-prometheus
python-dkimpy
python-enlighten
python-epub
python-events
python-fedmsg-meta-fedora-infrastructure
python-flask-wtf-decorators
python-flask-xml-rpc
python-flufl-i18n
python-fontMath
python-freeipa
python-gammu
python-genshi
python-genty
python-geojson
python-gerritlib
python-glyphsLib
python-gmqtt
python-hgapi
python-html5-parser
python-httpsig-cffi
python-hypothesis-fspaths
python-imagehash
python-ipgetter2
python-iptools
python-jack-client
python-jcconv
python-jinja2_pluralize
python-jinja2-time
python-journal-brief
python-json-minify
python-jsonpath-ng
python-jsonpath-rw
python-klusta
python-launchpadlib
python-lazr-uri
python-libusb1
python-lupa
python-markups
python-messaging
python-moksha-hub
python-murano-pkg-check
python-mutatormath
python-mwclient
python-neurosynth
python-num2words
python-oletools
python-openidc-client
python-oslo-context
python-oslo-metrics
python-os-service-types
python-ouimeaux
python-pdfkit
python-phonenumbers
python-poyo
python-pyasn1
python-pyclipper
python-pycryptodomex
python-pydispatcher
python-pyfim
python-pymediainfo
python-pyramid_sawing
python-pyramid-tm
python-pysignals
python-pysocks
python-pystray
python-pyte
python-pytest-fixture-config
python-pytest-flakes
python-pytest-runner
python-pytimeparse
python-repoze-lru
python-rmtest
python-rsd-lib
python-scripttest
python-serpy
python-shodan
python-simplegeneric
python-simplevisor
python-singledispatch
python-slacker
python-smartypants
python-sphinxcontrib-autoprogram
python-sphinxcontrib-trio
python-sql
python-ssdeep
python-statsd
python-svg-path
python-testscenarios
python-textile
python-textparser
python-tgscheduler
python-tidy
python-timeout-decorator
python-tosca-parser
python-translationstring
python-tree-format
python-typing-inspect
python-volatile
python-wadllib
python-whoosh
python-wsgi_intercept
python-www-authenticate
python-xmp-toolkit
python-xunitparser
python-yapsy
python-yaql
python-zc-customdoctests
python-zc-lockfile
python-zope-deprecation
python-zope-testing
python-zstd
sepolicy_analysis
tpm2-pkcs11
transifex-client

List of packages that actually failed in our testing copr but did not fail in a control copr:

ansible-inventory-grapher
btrfs-sxbackup
cepces
classification-banner
ddiskit
flent
fprettify
fros
gpgme
grammalecte
grpc
hddfancontrol
ioc-writer
lazygal
mopidy
mopidy-mpd
mozjs102
past-time
PyGreSQL
pyproject-rpm-macros
python3-lxc
python3-mallard-ducktype
python-aioresponses
python-binaryornot
python-bracex
python-case
python-cccolutils
python-chai
python-cliff
python-collada
python-colorclass
python-cpuinfo
python-cwcwidth
python-daemon
python-decorator
python-dfdatetime
python-digitalocean
python-distroinfo
python-django-contrib-comments
python-django-prometheus
python-epub
python-events
python-flake8-class-newline
python-flask-wtf-decorators
python-flask-xml-rpc
python-flufl-i18n
python-fontMath
python-freeipa
python-genshi
python-genty
python-geojson
python-hypothesis-fspaths
python-imagehash
python-iptools
python-jack-client
python-jcconv
python-jinja2_pluralize
python-jinja2-time
python-json-minify
python-jsonpath-ng
python-jsonpath-rw
python-jupyter-packaging
python-klusta
python-launchpadlib
python-lazr-uri
python-libusb1
python-lupa
python-markups
python-moksha-hub
python-mutatormath
python-mwclient
python-nose
python-oletools
python-openidc-client
python-oslo-context
python-oslo-metrics
python-ouimeaux
python-parsel
python-pdfkit
python-phonenumbers
python-poyo
python-pyasn1
python-pycdio
python-pyclipper
python-pydispatcher
python-pyemd
python-pygmtools
python-pymediainfo
python-pyproject-api
python-pyproject-hooks
python-pyramid_sawing
python-pyramid-tm
python-pysignals
python-pysrt
python-pytaglib
python-pyte
python-pytest-fixture-config
python-pytest-flakes
python-pytest-runner
python-pytimeparse
python-pyvmomi
python-repoze-lru
python-rmtest
python-robot-detection
python-scikit-build
python-scripttest
python-serpy
python-simplegeneric
python-singledispatch
python-slacker
python-sphinxcontrib-autoprogram
python-sphinxcontrib-trio
python-sql
python-ssdeep
python-statsd
python-svg-path
python-testscenarios
python-textparser
python-tgscheduler
python-timeout-decorator
python-translationstring
python-tree-format
python-trml2pdf
python-typing-inspect
python-virtualenv
python-volatile
python-wadllib
python-whoosh
python-wsgi_intercept
python-www-authenticate
python-xmp-toolkit
python-xunitparser
python-yaql
python-zc-customdoctests
python-zc-lockfile
python-zope-deprecation
python-zope-testing
python-zope-testrunner
python-zstd
sentencepiece
tpm2-pkcs11
transifex-client

Both lists together, by maintainers:

abbra      python-whoosh
abompard   python-flufl-i18n python-jinja2_pluralize python-pyramid_sawing python-pysocks python-translationstring python-zope-deprecation python-zope-testing python-zope-testrunner
adamwill   python-mwclient
adrian     flent
aekoroglu  python-django-prometheus python-pytaglib python-repoze-lru python-serpy python-xunitparser
ajax       binwalk
alexl      pycairo
amigadave  isrcsubmit python3-mallard-ducktype
amoralej   python-rsd-lib python-yaql
ankursinha python-klusta python-num2words python-pyemd python-pyfim
apevec     python-oslo-context python-testscenarios python-wsgi_intercept
asn        cepces
athoscr    python-booleanoperations python-epub python-fontMath python-glyphsLib python-mutatormath python-pyclipper
atim       kitty
aviso      python-enlighten
besser82   python-jcconv
caolanm    pycairo
carlwgeorge python-pyproject-hooks
chandankumar python-oslo-context python-tosca-parser python-wsgi_intercept
chedi      python-jinja2-time python-poyo python-pytest-flakes python-tree-format
chkr       calibre
churchyard pyproject-rpm-macros python-genty python-launchpadlib python-lazr-uri python-nose python-scripttest python-simplegeneric python-virtualenv python-wadllib
cicku      python-apsw
ckalina    ddiskit
cottsay    python-flake8-class-newline
cqi        python-django-contrib-comments
cstratak   python-virtualenv
dcallagh   python-phonenumbers python-www-authenticate
dcavalca   tpm2-pkcs11
dchen      cepces
defolos    grpc
dfateyev   python-apsw
dherrera   python-pytest-runner
dminer     python-zstd
dshea      python-jinja2_pluralize
echevemaster python-pydispatcher transifex-client
eeickmeyer python-jack-client
esyr       ddiskit
fab        ioc-writer past-time python-cx-oracle python-dfdatetime python-geojson python-gmqtt python-imagehash python-ipgetter2 python-jsonpath-ng python-jsonpath-rw python-shodan python-slacker python-textparser python-zc-customdoctests
fale       python-decorator python-pyvmomi
fed500     python-pygmtools
fedepell   python-jinja2-time
filiperosset hddfancontrol
fjanus     PyGreSQL
frantisekz mozjs102
frostyx    python-flask-wtf-decorators
fschwarz   python-digitalocean python-genshi
girst      mopidy mopidy-mpd
gsauthof   python-aioresponses
gui1ty     python-moksha-hub
hhorak     PyGreSQL
hiwkby     python-oslo-metrics
hobbes1069 python-collada
hvad       python-events
ignatenkobrain python-pytest-flakes python-simplegeneric
jamatos    python-markups python-smartypants
jcapitao   python-cliff python-distroinfo python-timeout-decorator
jcaratzas  python-repoze-lru python-simplegeneric python-singledispatch python-sphinxcontrib-trio python-zc-lockfile
jjelen     tpm2-pkcs11
jkucera    gpgme
jmontleon  python-dictdiffer
jonathanspw kitty python-freeipa python-parsel python-pystray python-pytest-runner python-zstd
jonny      python-libusb1
jorti      python-pysrt
jpena      python-murano-pkg-check python-tosca-parser
jruzicka   python-tosca-parser
jujens     python-CommonMark python-hgapi
jussilehtola fprettify
kalev      pycairo
kevin      calibre python-chai python-decorator python-html5-parser python-pycdio python-pyramid-tm python-pysocks python-pytest-fixture-config python-svg-path python-tgscheduler
kni        python-ouimeaux python-pysignals
ktdreyer   python-testscenarios
kushal     python-daemon
laxathom   python-gammu
lbalhar    python-jupyter-packaging python-pyproject-api python-pyproject-hooks python-virtualenv
lbazan     python-neurosynth transifex-client
lberk      python-rmtest
lbrabec    python-xunitparser
lcons      python-auth-credential python-dirq python-messaging python-simplevisor
lecris     python-scikit-build
limb       python-typing-inspect
lnie       libvirt-test-API
lyessaadi  python-yapsy
maha       python-sphinxcontrib-autoprogram
melmorabity grammalecte python-pycryptodomex
mfabian    nototools
mhayden    python-virtualenv
mhlavink   gpgme
mohanboddu python-openidc-client
mrc0mmand  gpgme
mrunge     python-pytimeparse python-singledispatch python-testscenarios python-virtualenv
nathans    python-rmtest
nb         python-digitalocean
neil       grpc python-gerritlib
ngompa     python-case python-dkimpy python-freeipa python-pystray python-robot-detection python-zstd
opohorel   python-launchpadlib python-lazr-uri python-wadllib
orion      python-virtualenv python-zope-testing
orphan     python-case python-fedmsg-meta-fedora-infrastructure python-hypothesis-fspaths python-iptools
osloup     PyGreSQL
pbrobinson adapt gtts gtts-token pyjokes python-can python-casttube tpm2-pkcs11
peter      python-libusb1
pingou     python-binaryornot python-case python-chai python-flask-xml-rpc
pkubat     PyGreSQL
pnemade    ansible-inventory-grapher python-bracex python-cpuinfo
poros      ddiskit
praiskup   PyGreSQL
puiterwijk python-cccolutils
pwhalen    pyjokes pyowm
pwu        nototools
qulogic    python-json-minify python-xmp-toolkit
radez      python-zc-lockfile python-zope-testing
raphgro    python-pdfkit python-pyvmomi
rathann    lazygal
rcallicotte python-pyvmomi
rcritten   cepces python-pyasn1
rdieter    gpgme python-mwclient
rebus      python-colorclass python-oletools
rhughes    pycairo
robert     python-oletools
rstrode    pycairo
sagitter   petsc
salimma    kitty python-lupa python-pytest-runner python-robot-detection python-volatile python-zope-testrunner python-zstd
sergiomb   python-gammu
sgallagh   python-whoosh
sharkcz    python-sql
shoracek   tpm2-pkcs11
slinabery  python-jsonpath-rw
solopasha  kitty
spichugi   python-pyasn1
suve       python-ssdeep
swt2c      binwalk
tdecacqu   python-statsd
terjeros   hgsvn python-cwcwidth python-httpsig-cffi python-pyte python-tidy
tflink     python-xunitparser
thebeanogamer classification-banner
thm        python-sphinxcontrib-trio python-textile python3-lxc
thofmann   btrfs-sxbackup
thrnciar   python-scikit-build
tieugene   python-trml2pdf
tohojo     flent
tomspur    python-daemon python-simplegeneric
topazus    sentencepiece
trawets    fros
ttorling   python-pysocks
twaugh     python-journal-brief
vascom     python-pymediainfo
vmojzis    sepolicy_analysis
vtrefny    python-pytimeparse
xavierb    python-daemon
ykarel     python-os-service-types
zaneb      python-autopage
zawertun   kitty
zbyszek    calibre
zsun       ddiskit

We will be slowly crushing the failures and opening bugzillas before we actually update setuptools.

Feedback

python-devel thread from August 2024: https://lists.fedoraproject.org/archives/list/python-devel@lists.fedoraproject.org/thread/YOC5527SRKNY2XG6CMS3KNGJ54O7SMXI/

Listed ~200 packages, ~30 were fixed in a month. Thank you, maintainers!

Benefit to Fedora

Fedora offers cutting-edge technologies for Python and setuptools is one of the most important Python packages. By doing this update we bring users the latest features that setuptools offers and also we enable Fedora 42+ to receive future updates as well.

Scope

• Proposal owners: update python-setuptools to 74.x.x or newer, open bugzillas, provide help

• Other developers: report problems to the upstream and backport patches to the affected packages. The impact can be tested using copr repository where setuptools 74+ has been built.

• Release engineering: #12323

• Policies and guidelines: N/A (not needed for this Change)

• Trademark approval: N/A (not needed for this Change)

• Alignment with the Fedora Strategy: First?

Upgrade/compatibility impact

There is a clean upgrade path from current version to python-setuptools 69.x.x. Fedora users using RPM-packaged setuptools will use setuptools 74+ by default.

Early Testing (Optional)

Do you require 'QA Blueprint' support? N

How To Test

• Find the package you want to update in the copr repository and check the build logs to determine the fail cause.
• Work with the upstream to resolve the issue or drop the need for setup.py test during the build of the package.
• When updating the package, you can test it using the same copr repository where the latest version of setuptools has been built.
• Let us know if something doesn't work as it should.

User Experience

Regular distro users shouldn't notice any change in setuptools behavior, except for upstream-intended changes.

Contingency Plan

• Contingency mechanism: (What to do? Who will do it?) Provide a compat package with setuptools 69
• Contingency deadline: mass rebuild
• Blocks release? No

Documentation

Setuptools changelog

This page is the documentation.