Talk:Features/Python 3.3

Scope of python3 in Fedora 18

Getting python3-using packages:

$ for f in $(repoquery --whatrequires python3) ; do echo $f ; repoquery --source $f ; done > sources.txt
$ grep src.rpm sources.txt | sort | uniq | wc -l
130

but it's showing some duplicates; adding filtering shows 105 src.rpms:

$ grep src.rpm sources.txt | python -c "import sys, re; print('\n'.join([re.match('(.*)-(.*)-(.*).src.rpm', line).group(1) for line in sys.stdin.readlines()]))"| sort |uniq
ATpy
babel
bpython
cobbler
deltarpm
dreampie
lcgdm
libselinux
libsemanage
loook
metagoofil
mpi4py
mysql-connector-python
nfoview
nordugrid-arc
numpy
pycmd
pyfits
pygobject3
pyke
pyp2rpm
pyparsing
PyQt4
pyshp
pytest
python3
python3-cairo
python3-chardet
python3-cherrypy
python3-dateutil
python3-postgresql
python3-zope-fixers
python-amqplib
python-ansi2html
python-anyjson
python-asciitable
python-auth-credential
python-beaker
python-BeautifulSoup
python-beautifulsoup4
python-cmd2
python-coverage
python-crypto
python-cvxopt
python-decorator
python-dirq
python-docutils
python-easygui
python-enchant
python-execnet
python-feedparser
python-flexmock
python-html5lib
python-httplib2
python-inotify
python-jinja2
python-lxml
python-mako
python-manuel
python-markdown
python-markdown2
python-markupsafe
python-messaging
python-mglob
python-minimock
python-mock
python-networkx
python-nose
python-nose1.1
python-pbs
python-pip
python-pkgwat-api
python-ply
python-psutil
python-psycopg2
python-py
python-pygments
python-pyro
python-requests
python-restauth
python-restauth-common
python-setproctitle
python-setuptools
python-simplegeneric
python-six
python-sleekxmpp
python-smbc
python-smbpasswd
python-sphinx
python-sqlalchemy
python-straight-plugin
python-tempita
python-tornado
python-urwid
python-zmq
python-zope-event
PyYAML
revisor
ris-linux
scipy
sip
stomppy
uwsgi
waf
znc

What rebuilds cleanly

After building "python3" from the "python3.3" branch, mockchain built the following successfully in order: (using this script to obtain the build order):

[david@surprise python3.3]$ grep Success localrepo/mockchain*.log |python -c"import sys, re; print '\n'.join([re.match('.*Success building (.+)', line).group(1) for line in sys.stdin.readlines()])"

cobbler-2.2.3-3.fc18.src.rpm
deltarpm-3.6-0.9.20110223git.fc18.src.rpm
dreampie-1.1.1-2.fc18.src.rpm
lcgdm-1.8.3.1-4.fc18.src.rpm
libselinux-2.1.11-4.fc18.src.rpm
libsemanage-2.1.8-4.fc18.src.rpm
loook-0.6.7-8.fc18.src.rpm
metagoofil-1.4b-5.fc18.src.rpm
mpi4py-1.3-3.fc18.src.rpm
mysql-connector-python-0.3.2-4.fc18.src.rpm
nordugrid-arc-2.0.0-3.fc18.src.rpm
pygpgme-0.3-2.fc18.src.rpm
pyke-1.1.1-8.fc18.src.rpm
pyparsing-1.5.6-4.fc18.src.rpm
python3-cairo-1.10.0-3.fc18.src.rpm
python3-chardet-2.0.1-7.fc18.src.rpm
python3-postgresql-1.0.2-3.fc18.src.rpm
python-asciitable-0.8.0-3.fc18.src.rpm
python-auth-credential-0.6-2.fc18.src.rpm
python-cmd2-0.6.4-3.fc18.src.rpm
python-crypto-2.6-2.fc18.src.rpm
python-dirq-1.2.2-2.fc18.src.rpm
python-httplib2-0.7.4-5.fc18.src.rpm
python-inotify-0.9.3-2.fc18.src.rpm
python-messaging-0.10-2.fc18.src.rpm
python-mock-0.8.0-3.fc18.src.rpm
python-pbs-4.3.3-2.fc18.src.rpm
python-ply-3.4-2.fc18.src.rpm
python-psutil-0.5.1-2.fc18.src.rpm
python-requests-0.13.1-2.fc18.src.rpm
python-restauth-common-0.5.1-5.fc18.src.rpm
python-restauth-0.5.1-5.fc18.src.rpm
python-six-1.1.0-3.fc18.src.rpm
python-sleekxmpp-1.0-0.10.beta2.fc18.src.rpm
python-straight-plugin-1.1.1-0.9.20111110.git57ef11c.fc18.src.rpm
revisor-2.2-5.fc18.src.rpm
ris-linux-0.4-10.fc18.src.rpm
sip-4.13.3-2.fc18.src.rpm
stomppy-3.0.5-2.fc18.src.rpm
uwsgi-1.2.4-2.fc18.src.rpm
waf-1.7.0-1.fc18.1.src.rpm
pygobject3-3.3.4-1.fc18.src.rpm
PyQt4-4.9.4-2.fc18.src.rpm
python-setuptools-0.6.28-1.fc18.src.rpm
ATpy-0.9.6-2.fc18.src.rpm
babel-0.9.6-3.fc18.src.rpm
bpython-0.11-2.fc18.src.rpm
nfoview-1.10-4.fc18.src.rpm
pyshp-1.1.4-4.fc18.src.rpm
python3-cherrypy-3.2.0-0.rc1.r2567.1.fc18.5.src.rpm
python3-dateutil-2.0-3.fc18.src.rpm
python3-zope-fixers-1.0-3.fc18.src.rpm
python-anyjson-0.3.1-4.fc18.src.rpm
python-coverage-3.5.2-0.2.b1.fc18.src.rpm
python-cvxopt-1.1.5-3.fc18.src.rpm
python-decorator-3.3.3-2.fc18.src.rpm
python-enchant-1.6.5-6.fc18.src.rpm
python-feedparser-5.1.2-3.fc18.src.rpm
python-flexmock-0.9.4-2.fc18.src.rpm
python-html5lib-0.95-2.fc18.src.rpm
python-lxml-2.3.3-2.fc18.src.rpm
python-markupsafe-0.11-5.fc18.src.rpm
python-minimock-1.2.7-3.fc18.src.rpm
python-nose-cover3-0.1.0-3.fc18.src.rpm
python-pip-1.1-2.fc18.src.rpm
python-pkgwat-api-0.3-4.fc18.src.rpm
python-profilehooks-1.6-2.fcf18.src.rpm
python-pyro-4.9-3.fc18.src.rpm
python-simplegeneric-0.8-4.fc18.src.rpm
python-tempita-0.5.1-2.fc18.src.rpm
python-tornado-2.2.1-2.fc18.src.rpm
python-urwid-1.0.0-4.fc18.src.rpm
python-zope-event-3.5.1-3.fc18.src.rpm
PyYAML-3.10-4.fc18.src.rpm
python-beautifulsoup4-4.1.1-4.fc18.src.rpm
python-BeautifulSoup-3.2.1-2.fc18.src.rpm
python-jinja2-2.6-3.fc18.src.rpm
python-easygui-0.96-4.fc18.src.rpm
python-smbc-1.0.13-3.fc18.src.rpm
python-smbpasswd-1.0.1-20.fc18.src.rpm
python-py-1.4.9-2.fc18.src.rpm
pycmd-1.0-6.fc18.src.rpm
pytest-2.2.4-2.fc18.src.rpm
python-execnet-1.1-2.fc18.src.rpm
python-nose-1.1.2-3.fc18.src.rpm
python-ansi2html-0.9.1-6.fc18.src.rpm
python-beaker-1.5.4-4.fc18.src.rpm
python-mako-0.5.0-3.fc18.src.rpm
python-markdown-2.2.0-2.fc18.src.rpm
python-pygments-1.4-5.fc18.src.rpm
python-markdown2-2.0.0-2.fc18.src.rpm

Fixes that were needed

Intermittent failures

I had to apply this fix for a localhost-using test in python-setuptools; this change isn't in git yet:

[david@surprise python-setuptools]$ git diff
diff --git a/python-setuptools.spec b/python-setuptools.spec
index 1dc1a9a..db84cb3 100644
--- a/python-setuptools.spec
+++ b/python-setuptools.spec
@@ -21,7 +21,7 @@ Source2:        zpl.txt
 Patch0: distribute-different-exception-message.patch
 # Sometimes this times out in the build system.  Hanging onto the patch in git
 # for a bit in case that behavior returns
-#Patch1: distribute-timeout-exception.patch
+Patch1: distribute-timeout-exception.patch
 
 BuildRoot:      %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
 
@@ -68,7 +68,7 @@ This package contains the distribute fork of setuptools.
 %setup -q -n %{srcname}-%{version}
 
 %patch0 -p1 -b .excmsg
-#patch1 -p1 -b .exctype
+%patch1 -p1 -b .exctype
 
 find -name '*.txt' | xargs chmod -x
 find . -name '*.orig' -exec rm \{\} \;

Packaging tweaks

 DONE: python-easygui/python-easygui-0.96-3.fc18.src.rpm
   RPM build errors:
     File not found by glob: /builddir/build/BUILDROOT/python-easygui-0.96-3.fc18.x86_64/usr/lib/python3.3/site-packages/__pycache__/easygui.cpython-32.py*
   Test built in local mock; fix pushed to git master
 DONE: python-smbc/python-smbc-1.0.13-2.fc18.src.rpm
   RPM build errors:
     File not found by glob: /builddir/build/BUILDROOT/python-smbc-1.0.13-2.fc18.x86_64/usr/lib64/python3.3/site-packages/smbc.cpython-3?mu.so
   Test built in local mock; fix pushed to git master
 DONE: python-smbpasswd/python-smbpasswd-1.0.1-19.fc18.src.rpm
    RPM build errors:
      File not found by glob: /builddir/build/BUILDROOT/python-smbpasswd-1.0.1-19.fc18.x86_64/usr/lib64/python3.3/site-packages/smbpasswd.cpython-3?mu.so
   Test built in local mock; fix pushed to git master

Breaking a circular dep

python-py.spec has:

# we have a circular (build) dependency with the (new) pytest package
# when generating the docs or running the testsuite
%global with_docs 1
%global run_check 1

Disabling both, regening python-py src.rpm and trying again:

[david@surprise python-py]$ git diff
diff --git a/python-py.spec b/python-py.spec
index ef7ad93..8f0eb69 100644
--- a/python-py.spec
+++ b/python-py.spec
@@ -6,8 +6,8 @@
 
 # we have a circular (build) dependency with the (new) pytest package
 # when generating the docs or running the testsuite
-%global with_docs 1
-%global run_check 1
+%global with_docs 0
+%global run_check 0
 
 %global pytest_version 2.2.4

Temporary Fixes

python-nose

I turned off the %check in python-nose in order to build the deps before having the full selftest suite for nose itself passing.

Remaining work

Packaging Tweaks

python-psycopg2/python-psycopg2-2.4.5-2.fc18.src.rpm

    + for python in python python-debug python3 python3.2dmu
    + python3.2dmu setup.py build
    /var/tmp/rpm-tmp.ZyG8xO: line 34: python3.2dmu: command not found

pyp2rpm/pyp2rpm-0.5.1-2.fc18.src.rpm

py.test-3.2: command not found

python-amqplib/python-amqplib-1.0.2-4.fc18.src.rpm

nosetests-3.2: command not found

Selftest failures

TODO: python-docutils/python-docutils-0.9.1-1.fc18.src.rpm

tests fail: ImportError: No module named 'en'

TODO:python-nose1.1/python-nose1.1-1.1.2-6.fc18.src.rpm

selftest failures: FAILED (SKIP=9, errors=1, failures=3)

TODO: python-nose/python-nose-1.1.2-3.fc18.src.rpm

selftest failures: FAILED (SKIP=9, errors=1, failures=3)

TODO: python-cement/python-cement-1.9.14-2.fc18.src.rpm

This happen in 4 places in the test suite:

    File "/builddir/build/BUILD/python3-python-cement-1.9.14-2.fc18/cement/core/extension.py", line 141, in load_extension
      __import__(ext_module, globals(), locals(), [], -1)
  ValueError: level must be >= 0

Unicode issues

TODO: numpy/numpy-1.6.2-2.fc18.src.rpm

Needs porting: has various things that use internals of unicode Upstream status?

 http://projects.scipy.org/numpy/ticket/1471
 (also http://projects.scipy.org/numpy/ticket/2145 )
 Discussed here: http://www.mail-archive.com/numpy-discussion@scipy.org/msg38344.html

Missing build deps

  TODO: pyfits/pyfits-3.0.8-2.fc18.src.rpm
    BR: numpy
  TODO: python-manuel/python-manuel-1.6.0-2.fc18.src.rpm
    BR: python3-sphinx
  TODO: python-networkx/python-networkx-1.6-3.fc18.src.rpm
    Many BRs
  TODO: python-setproctitle/python-setproctitle-1.1.3-3.fc18.src.rpm
    BR: python3-nose
  TODO: python-sphinx/python-sphinx-1.1.3-2.fc18.src.rpm
    BR: python3-nose, python3-pygments
  TODO: python-sqlalchemy/python-sqlalchemy-0.7.8-2.fc18.src.rpm
    BR: python3-nose
  TODO: python-zmq/python-zmq-2.2.0-2.fc18.src.rpm
    BR: python3-nose
  TODO: scipy/scipy-0.10.1-2.fc18.src.rpm
    BR: python3-numpy
    BR: python3-nose

SWIG issues

TODO: znc/znc-0.206-2.fc18.src.rpm

swig errors