Multiple Python configurations
What might we want to vary?
"Debug" versus "standard" build
Python can be built with various debugging options, typically just used when working on Python itself. In my experience "--with-pydebug" roughly halves the speed of the interpreter, but adds lots of extra checking (e.g. reference leaks). It also changes the ABI.
We could produce an entirely separate "debug" python stack, with various debugging checks configured in (breaking ABI, and slowing things down); hope was to make it trivial to run /usr/bin/python2.6-debug (provided you also have the debug build of all of the modules, etc; doesn't work, you need -debug of full stack, I think; you'd have a yum-debug, with -debug of all the stack).
| Configuration option | Definition | Effect |
|---|---|---|
| --with-pydebug | Py_DEBUG | Implies ABI-breaking other options (see below), and adds additional self-tests throughout libpython |
| Py_TRACE_REFS | (implied by Py_DEBUG): Track a doubly-linked list of all objects. Changes libpython ABI: adds _ob_prev and _ob_next members to PyObject | |
| Py_REF_DEBUG | (implied by Py_TRACE_REFS); track total reference count of all objects (_Py_RefTotal), and ensure it never goes negative | |
| --without-pymalloc | Avoid dealing with the arena allocator: go straight to malloc/free; changes API meaning | |
| COUNT_ALLOCS | Track the number of allocations made per PyTypeObject; changes API meaning | |
| --with-tsc | WITH_TSC | enable/disable timestamp counter profile |
| DYNAMIC_EXECUTION_PROFILE | Track how many times each opcode is executed | |
| DXPAIRS | Track how many times each pair of opcodes is executed | |
| LLTRACE | Track all stack operations | |
| USE_STACKCHECK |
(See pyconfig.h)
--with-valgrind (not needed if --without-pymalloc)
--enable-profiling ?
Future work: we may want to have with/without LLVM as a build option
"--with-pydebug" is the big one, implying various ABI-breaking changes.
Might want to turn off optimizations, or only optimize lightly, whilst we're at it (e.g. "-Os" ?)
Would need to do as a Feature. Do it for python3 as well, potentially giving 4 subpackages per build.
So for package "coverage" (with C extension modules) you might have the python-coverage.src.rpm emit the following subpackages as part of its build:
- python-coverage (or "python2-coverage"?)
- python-dbg-coverage (or "python2-dbg-coverage"?)
- python3-coverage
- python3-dbg-coverage
- python-coverage-debuginfo
whereas for package "setuptools" (without C extension modules) you might have the python-setuptools.src.rpm emit the following subpackages:
- python-setuptools (or "python2-setuptools"?)
- python3-setuptools
UCS2 vs UCS4
Is it sane to double the number of configurations, offering UCS2 vs UCS4? I don't think so, but listing it here for completeness. I'm not convinced that the choice of UCS4 is correct; are we merely doing it due to historical precedent?
Multiple minor versions
- advance versions of 2.7 and 3.2
Alternate Implementations
Can we use the following to build out multiple stacks? (e.g. share pure-python modules between Jython and CPython?)
- Unladen Swallow
- Jython
- PyPy
Dealing with multiple python implementations
What would packaging guidelines look like? Generalization of current guidelines to add additional builds. See my notes sent to the packaging list before (and roundly rejected) Too much work for package maintainers?
Other things we could change
dist-packages rather than site-packages ?
Debian uses "dist-packages" for things coming as part of the distribution, to avoid PyPi users splatting "site-packages"; should we? It seems more logical to me, keeping site-packages for site-specific use.