From Fedora Project Wiki

No more automagic Python bytecompilation (phase 3)

Summary

See Changes/No_more_automagic_Python_bytecompilation and Changes/No_more_automagic_Python_bytecompilation_phase_2. Now, %global _python_bytecompile_extra 1 won't be allowed anymore and raises an error with a link to this change.

Owner

Current status

Detailed Description

See the Detailed Description of the previous Change Proposal and the Detailed Description of its phase 2.

Currently, Fedora rawhide contains ~130 packages with %global _python_bytecompile_extra 1 in their specfiles but surprisingly only 42 of them actually ship any .pyc files outside the standard location /usr/lib(64)?/python[0-9]\.[0-9]+. That might be caused by either of the following:

  • there is nothing to byte-compile — the statement is a leftover to be removed
  • The automagic bytecompilation uses /usr/bin/python by default (for historical reasons) but /usr/bin/python is not present in the buildroot by default.

Our plan is to finish the announced removal of this automagic Python bytecompilation, remove it from packages where it's not necessary and fix it with %py_byte_compile for packages where it's needed.

The documentation already contains the following paragraph: 

If you have *.py files outside of the /usr/lib(64)?/pythonX.Y/ directories and you require those files to be byte compiled (e.g. it’s an importable Python module) you MUST compile them explicitly using the %py_byte_compile macro. Note that not all Python files are importable Python modules; when in doubt, grep the sources for the appropriate import statement.

so no changes have to be made there.

Affected packages

From 2020-05-20. 

bugzilla
calamares
ceph
chromium
cinnamon-screensaver
edk2
eog-plugins
fish
fleet-commander-admin
fleet-commander-client
freecad
gaupol
gazebo
gdb
gedit
gedit-latex
gedit-plugins
git-cola
glusterfs
gnome-code-assistance
gnome-devel-docs
grass
gtk-doc
gtranslator
ibus
ibus-anthy
ibus-hangul
ibus-libpinyin
ibus-libzhuyin
ibus-pinyin
ibus-table
ibus-typing-booster
ibus-uniemoji
insight
kajongg
kdevelop-python
klatexformula
libpeas
libreoffice
libsmbios
libunity
lirc
lyx
mate-menu
mingw-glib2
mirrormanager2
odcs
onboard
otf2
paraview
pcs
php-opencloud-openstack
pygobject2
pygtk2
python-cherrypy
python-flask-silk
python-genshi
python-pycurl
python-reportlab
qpid-dispatch
rhythmbox
sems
sigul
soundconverter
sugar
sugar-abacus
sugar-browse
sugar-calculator
sugar-castle
sugar-chat
sugar-clock
sugar-colordeducto
sugar-countries
sugar-deducto
sugar-distance
sugar-finance
sugar-flip
sugar-flipsticks
sugar-fototoon
sugar-fractionbounce
sugar-getiabooks
sugar-imageviewer
sugar-implode
sugar-infoslicer
sugar-jukebox
sugar-kuku
sugar-labyrinth
sugar-locosugar
sugar-log
sugar-maze
sugar-measure
sugar-memorize
sugar-moon
sugar-nutrition
sugar-paint
sugar-physics
sugar-pippy
sugar-playgo
sugar-portfolio
sugar-pukllanapac
sugar-read
sugar-recall
sugar-record
sugar-ruler
sugar-speak
sugar-srilanka
sugar-starchart
sugar-stopwatch
sugar-story
sugar-terminal
sugar-turtleart
sugar-typing-turtle
sugar-view-slides
sugar-visualmatch
sugar-words
sugar-write
sugar-xoeditor
sugar-xoirc
sugar-yupana
synfigstudio
system-config-repo
system-switch-java
system-switch-mail
texlive
totem
transmageddon
ufw-kde
variety
virt-manager
vtk
xed

EPEL compatibility

Explicit compilation with %py_byte_compile is EPEL 6/7/8 compatible.

In EPEL 6 and EPEL 7, the brp-python-bytecompile script compiles all "extra" directories with %__python (if it is available in the buildroot, this happens always, there is no opt out other than disabling the entire script). The %_python_bytecompile_extra value has no meaning and is not used. EPEL 6/7 compatible spec files adapted to work in Fedora 33+ (via the recommended way outline in this proposal) will continue to be EPEL 6/7 compatible (regarding Python byte compilation of "extra" directories). In the worst case scenario, some files will be byte compiled twice (explicitly, then automagically).

In EPEL 8, the "extra" directories are not compiled by default, but they are compiled with %__python if %_python_bytecompile_extra is set to 1 (same as in e.g. Fedora 32 or Fedora 33 prior to this chnage). Once this starts to be an error in Fedora 33+, spec files brought from EPEL 8 with %global _python_bytecompile_extra 1 to Fedora will need to be adapted. EPEL 8 compatible spec files adapted to work in Fedora 33+ (via the recommended way outline in this proposal) will continue to be EPEL 8 compatible (regarding Python byte compilation of "extra" directories).

Benefit to Fedora

More explicit specfiles when it comes to Python byte compilation. This change also supports the dropping of unversioned %{__python} macro.

Scope

  • Other developers: Maintainers of the mentioned python packages will have to change their packages to use explicit %py_byte_compile
  • Release engineering: No Release engineering work is needed for this change
  • Policies and guidelines: nothing to be changed
  • Trademark approval: not needed

Upgrade/compatibility impact

None expected.

How To Test

Build a package with %global _python_bytecompile_extra 1. The build should fail with an actionable error message.

User Experience

The users of this change are packagers. The new behavior should make byte-compilation more obvious, explicit, and discoverable. Users of Fedora should not feel this (except if this change uncovers a packaging bug).

Contingency Plan

  • Contingency mechanism: revert the changes in redhat-rpm-config
  • Contingency deadline: none (not a System Wide Change)
  • Blocks release? no (not a System Wide Change)
  • Blocks product? no

Documentation

The guidelines already contain up-to-date documentation.

Release Notes

This change does not deserve Release Notes, it is not user-facing.

Retrieved from "https://fedoraproject.org/w/index.php?title=Changes/No_more_automagic_Python_bytecompilation_phase_3&oldid=579398"