Talk:Features/Python 3.3

From FedoraProject

(Difference between revisions)
Jump to: navigation, search
(python-docutils)
(release notes: new section)
 
(12 intermediate revisions by one user not shown)
Line 3: Line 3:
  
 
Dependency chain of the builds is:
 
Dependency chain of the builds is:
* python-docutils
+
* python-sphinx
** python-sphinx
+
** python-execnet
*** python-execnet
+
** python-manuel
*** python-manuel
+
 
* scipy
 
* scipy
 
** python-networkx
 
** python-networkx
 
* gcc-python-plugin
 
* gcc-python-plugin
 
== python-docutils ==
 
Tests fail (against 3.3b2): (failures=1, errors=4)
 
 
3 failures in test_writers.test_odt.DocutilsOdtTestCase: with "TypeError: expected an Element, not _ElementInterfaceWrapper"
 
odf writer fails when run with Python3.3b1/b2: filed as https://sourceforge.net/tracker/?func=detail&aid=3555164&group_id=38414&atid=422030
 
 
Other failures:
 
<pre>
 
======================================================================
 
ERROR: test_unicode (test_error_reporting.ErrorStringTests)
 
----------------------------------------------------------------------
 
Traceback (most recent call last):
 
  File "/home/david/coding/dist-git-fedora/python-docutils/python3-python-docutils-0.10-0.3.20120730svn7490.fc19/test3/test_error_reporting.py", line 154, in test_unicode
 
    str(ErrorString(ImportError(self.bs))))
 
  File "/home/david/coding/dist-git-fedora/python-docutils/python3-python-docutils-0.10-0.3.20120730svn7490.fc19/build/lib/docutils/utils/error_reporting.py", line 123, in __str__
 
    super(ErrorString, self).__str__())
 
  File "/home/david/coding/dist-git-fedora/python-docutils/python3-python-docutils-0.10-0.3.20120730svn7490.fc19/build/lib/docutils/utils/error_reporting.py", line 73, in __str__
 
    return str(self.data)
 
TypeError: __str__ returned non-string (type bytes)
 
</pre>
 
 
This one looks like a hash-randomization dict ordering issue:
 
<pre>
 
test_functional.py: functional/tests/standalone_rst_xetex.py; test (test_functional.FunctionalTestCase):
 
--- functional/expected/standalone_rst_xetex.tex
 
+++ functional/output/standalone_rst_xetex.tex
 
@@ -7,7 +7,7 @@
 
\usepackage{amsmath}
 
\usepackage{polyglossia}
 
\setdefaultlanguage{english}
 
-\setotherlanguages{german,british,french}
 
+\setotherlanguages{german,french,british}
 
\usepackage{color}
 
\usepackage{float} % float configuration
 
\floatplacement{figure}{H} % place figures here definitely
 
</pre>
 
  
 
== python-manuel ==
 
== python-manuel ==
BR: python3-sphinx
+
https://admin.fedoraproject.org/updates/python-manuel-1.6.0-3.fc18
 
+
== python-nose1.1 ==
+
selftest failures: FAILED (SKIP=9, errors=1, failures=3)
+
  
 
== python-sphinx ==
 
== python-sphinx ==
BR: python3-docutils
+
https://admin.fedoraproject.org/updates/FEDORA-2012-12707/python-sphinx-1.1.3-5.fc18
  
 
== python-execnet ==
 
== python-execnet ==
BR: python3-sphinx
+
[http://koji.fedoraproject.org/koji/buildinfo?buildID=346097 python-execnet-1.1-3.fc18] built against 3.3
  
 
== scipy ==
 
== scipy ==
Line 74: Line 33:
  
 
== gcc-python-plugin ==
 
== gcc-python-plugin ==
(in progress)
+
https://admin.fedoraproject.org/updates/gcc-python-plugin-0.9-4.fc18
  
 
= Packages that have been rebuilt in koji =
 
= Packages that have been rebuilt in koji =
Line 179: Line 138:
 
python-setproctitle
 
python-setproctitle
 
python-nose1.1
 
python-nose1.1
 +
python-docutils
 
</pre>
 
</pre>
 
 
  
 
= Scope of python3 in Fedora 18 =
 
= Scope of python3 in Fedora 18 =
Line 519: Line 477:
 
== python-nose ==
 
== python-nose ==
 
I turned off the %check in python-nose in order to build the deps before having the full selftest suite for nose itself passing.
 
I turned off the %check in python-nose in order to build the deps before having the full selftest suite for nose itself passing.
 +
 +
== python-docutils ==
 +
Tests fail (against 3.3b2): (failures=1, errors=4)
 +
 +
3 failures in test_writers.test_odt.DocutilsOdtTestCase: with "TypeError: expected an Element, not _ElementInterfaceWrapper"
 +
odf writer fails when run with Python3.3b1/b2: filed as https://sourceforge.net/tracker/?func=detail&aid=3555164&group_id=38414&atid=422030
 +
 +
Hash-randomization dict ordering issue; sent patch upstream as http://sourceforge.net/tracker/?func=detail&aid=3555160&group_id=38414&atid=422032
 +
 +
Other failures reported as https://sourceforge.net/tracker/?func=detail&aid=3561133&group_id=38414&atid=422030
  
 
= Remaining work =
 
= Remaining work =
Line 525: Line 493:
 
=== TODO: znc/znc-0.206-2.fc18.src.rpm ===  
 
=== TODO: znc/znc-0.206-2.fc18.src.rpm ===  
 
swig errors
 
swig errors
 +
 +
== release notes ==
 +
 +
added to release notes at https://fedoraproject.org/wiki/Documentation_Development_Beat [[User:Immanetize|Immanetize]] ([[User talk:Immanetize|talk]]) 17:21, 26 October 2012 (UTC)

Latest revision as of 17:21, 26 October 2012

Contents

[edit] Packages that have not yet been rebuilt in koji

I'm working on these. See notes below

Dependency chain of the builds is:

  • python-sphinx
    • python-execnet
    • python-manuel
  • scipy
    • python-networkx
  • gcc-python-plugin

[edit] python-manuel

https://admin.fedoraproject.org/updates/python-manuel-1.6.0-3.fc18

[edit] python-sphinx

https://admin.fedoraproject.org/updates/FEDORA-2012-12707/python-sphinx-1.1.3-5.fc18

[edit] python-execnet

python-execnet-1.1-3.fc18 built against 3.3

[edit] scipy

Cython sources need regenerating

  File "orthogonal_eval.pyx", line 93, in init scipy.special.orthogonal_eval (scipy/special/orthogonal_eval.c:4126)
ValueError: level must be >= 0

(also some swig issues, I think)


[edit] python-networkx

BR: python3-scipy

[edit] gcc-python-plugin

https://admin.fedoraproject.org/updates/gcc-python-plugin-0.9-4.fc18

[edit] Packages that have been rebuilt in koji

python3
python-setuptools
deltarpm
dreampie
lcgdm
libselinux
libsemanage
loook
metagoofil
mpi4py
mysql-connector-python
nordugrid-arc
pygpgme
pyke
pyparsing
python3-cairo
python3-chardet
python3-postgresql
python-asciitable
python-auth-credential
python-cmd2
python-crypto
python-dirq
python-httplib2
python-inotify
python-messaging
python-mock
python-pbs
python-ply
python-psutil
python-requests
python-restauth-common
python-restauth
python-six
python-sleekxmpp
python-straight-plugin
revisor
ris-linux
sip
stomppy
uwsgi
waf
pygobject3
PyQt4
ATpy
babel
bpython
nfoview
pyshp
python3-cherrypy
python3-dateutil
python3-zope-fixers
python-anyjson
python-coverage
python-cvxopt
python-decorator
python-enchant
python-feedparser
python-flexmock
python-html5lib
python-lxml
python-markupsafe
python-minimock
python-nose-cover3
python-pip
python-pkgwat-api
python-profilehooks
python-pyro
python-simplegeneric
python-tempita
python-tornado
python-urwid
python-zope-event
PyYAML
python-beautifulsoup4
python-BeautifulSoup
python-jinja2
python-easygui
python-smbc
python-smbpasswd
python-py
pycmd
autoarchive
python-psycopg2
python-mglob
python-nose
python-beaker
python-markdown
python-pygments
python-amqplib
python-cement
python-zmq
python-mako
python-markdown2
pytest
numpy
pyfits
pyp2rpm
python-ansi2html
python-setproctitle
python-nose1.1
python-docutils

[edit] Scope of python3 in Fedora 18

Getting python3-using packages:

$ for f in $(repoquery --whatrequires python3) ; do echo $f ; repoquery --source $f ; done > sources.txt
$ grep src.rpm sources.txt | sort | uniq | wc -l
130

but it's showing some duplicates; adding filtering shows 105 src.rpms:

$ grep src.rpm sources.txt | python -c "import sys, re; print('\n'.join([re.match('(.*)-(.*)-(.*).src.rpm', line).group(1) for line in sys.stdin.readlines()]))"| sort |uniq
ATpy
babel
bpython
cobbler
deltarpm
dreampie
lcgdm
libselinux
libsemanage
loook
metagoofil
mpi4py
mysql-connector-python
nfoview
nordugrid-arc
numpy
pycmd
pyfits
pygobject3
pyke
pyp2rpm
pyparsing
PyQt4
pyshp
pytest
python3
python3-cairo
python3-chardet
python3-cherrypy
python3-dateutil
python3-postgresql
python3-zope-fixers
python-amqplib
python-ansi2html
python-anyjson
python-asciitable
python-auth-credential
python-beaker
python-BeautifulSoup
python-beautifulsoup4
python-cmd2
python-coverage
python-crypto
python-cvxopt
python-decorator
python-dirq
python-docutils
python-easygui
python-enchant
python-execnet
python-feedparser
python-flexmock
python-html5lib
python-httplib2
python-inotify
python-jinja2
python-lxml
python-mako
python-manuel
python-markdown
python-markdown2
python-markupsafe
python-messaging
python-mglob
python-minimock
python-mock
python-networkx
python-nose
python-nose1.1
python-pbs
python-pip
python-pkgwat-api
python-ply
python-psutil
python-psycopg2
python-py
python-pygments
python-pyro
python-requests
python-restauth
python-restauth-common
python-setproctitle
python-setuptools
python-simplegeneric
python-six
python-sleekxmpp
python-smbc
python-smbpasswd
python-sphinx
python-sqlalchemy
python-straight-plugin
python-tempita
python-tornado
python-urwid
python-zmq
python-zope-event
PyYAML
revisor
ris-linux
scipy
sip
stomppy
uwsgi
waf
znc

[edit] What rebuilds cleanly

After building "python3" from the "python3.3" branch, mockchain built the following successfully in order: (using this script to obtain the build order):

[david@surprise python3.3]$ grep Success localrepo/mockchain*.log |python -c"import sys, re; print '\n'.join([re.match('.*Success building (.+)', line).group(1) for line in sys.stdin.readlines()])"
cobbler-2.2.3-3.fc18.src.rpm
deltarpm-3.6-0.9.20110223git.fc18.src.rpm
dreampie-1.1.1-2.fc18.src.rpm
lcgdm-1.8.3.1-4.fc18.src.rpm
libselinux-2.1.11-4.fc18.src.rpm
libsemanage-2.1.8-4.fc18.src.rpm
loook-0.6.7-8.fc18.src.rpm
metagoofil-1.4b-5.fc18.src.rpm
mpi4py-1.3-3.fc18.src.rpm
mysql-connector-python-0.3.2-4.fc18.src.rpm
nordugrid-arc-2.0.0-3.fc18.src.rpm
pygpgme-0.3-2.fc18.src.rpm
pyke-1.1.1-8.fc18.src.rpm
pyparsing-1.5.6-4.fc18.src.rpm
python3-cairo-1.10.0-3.fc18.src.rpm
python3-chardet-2.0.1-7.fc18.src.rpm
python3-postgresql-1.0.2-3.fc18.src.rpm
python-asciitable-0.8.0-3.fc18.src.rpm
python-auth-credential-0.6-2.fc18.src.rpm
python-cmd2-0.6.4-3.fc18.src.rpm
python-crypto-2.6-2.fc18.src.rpm
python-dirq-1.2.2-2.fc18.src.rpm
python-httplib2-0.7.4-5.fc18.src.rpm
python-inotify-0.9.3-2.fc18.src.rpm
python-messaging-0.10-2.fc18.src.rpm
python-mock-0.8.0-3.fc18.src.rpm
python-pbs-4.3.3-2.fc18.src.rpm
python-ply-3.4-2.fc18.src.rpm
python-psutil-0.5.1-2.fc18.src.rpm
python-requests-0.13.1-2.fc18.src.rpm
python-restauth-common-0.5.1-5.fc18.src.rpm
python-restauth-0.5.1-5.fc18.src.rpm
python-six-1.1.0-3.fc18.src.rpm
python-sleekxmpp-1.0-0.10.beta2.fc18.src.rpm
python-straight-plugin-1.1.1-0.9.20111110.git57ef11c.fc18.src.rpm
revisor-2.2-5.fc18.src.rpm
ris-linux-0.4-10.fc18.src.rpm
sip-4.13.3-2.fc18.src.rpm
stomppy-3.0.5-2.fc18.src.rpm
uwsgi-1.2.4-2.fc18.src.rpm
waf-1.7.0-1.fc18.1.src.rpm
pygobject3-3.3.4-1.fc18.src.rpm
PyQt4-4.9.4-2.fc18.src.rpm
python-setuptools-0.6.28-1.fc18.src.rpm
ATpy-0.9.6-2.fc18.src.rpm
babel-0.9.6-3.fc18.src.rpm
bpython-0.11-2.fc18.src.rpm
nfoview-1.10-4.fc18.src.rpm
pyshp-1.1.4-4.fc18.src.rpm
python3-cherrypy-3.2.0-0.rc1.r2567.1.fc18.5.src.rpm
python3-dateutil-2.0-3.fc18.src.rpm
python3-zope-fixers-1.0-3.fc18.src.rpm
python-anyjson-0.3.1-4.fc18.src.rpm
python-coverage-3.5.2-0.2.b1.fc18.src.rpm
python-cvxopt-1.1.5-3.fc18.src.rpm
python-decorator-3.3.3-2.fc18.src.rpm
python-enchant-1.6.5-6.fc18.src.rpm
python-feedparser-5.1.2-3.fc18.src.rpm
python-flexmock-0.9.4-2.fc18.src.rpm
python-html5lib-0.95-2.fc18.src.rpm
python-lxml-2.3.3-2.fc18.src.rpm
python-markupsafe-0.11-5.fc18.src.rpm
python-minimock-1.2.7-3.fc18.src.rpm
python-nose-cover3-0.1.0-3.fc18.src.rpm
python-pip-1.1-2.fc18.src.rpm
python-pkgwat-api-0.3-4.fc18.src.rpm
python-profilehooks-1.6-2.fcf18.src.rpm
python-pyro-4.9-3.fc18.src.rpm
python-simplegeneric-0.8-4.fc18.src.rpm
python-tempita-0.5.1-2.fc18.src.rpm
python-tornado-2.2.1-2.fc18.src.rpm
python-urwid-1.0.0-4.fc18.src.rpm
python-zope-event-3.5.1-3.fc18.src.rpm
PyYAML-3.10-4.fc18.src.rpm
python-beautifulsoup4-4.1.1-4.fc18.src.rpm
python-BeautifulSoup-3.2.1-2.fc18.src.rpm
python-jinja2-2.6-3.fc18.src.rpm
python-easygui-0.96-4.fc18.src.rpm
python-smbc-1.0.13-3.fc18.src.rpm
python-smbpasswd-1.0.1-20.fc18.src.rpm
python-py-1.4.9-2.fc18.src.rpm
pycmd-1.0-6.fc18.src.rpm
pytest-2.2.4-2.fc18.src.rpm
python-execnet-1.1-2.fc18.src.rpm
python-nose-1.1.2-3.fc18.src.rpm
python-ansi2html-0.9.1-6.fc18.src.rpm
python-beaker-1.5.4-4.fc18.src.rpm
python-mako-0.5.0-3.fc18.src.rpm
python-markdown-2.2.0-2.fc18.src.rpm
python-pygments-1.4-5.fc18.src.rpm
python-markdown2-2.0.0-2.fc18.src.rpm
python-amqplib-1.0.2-5.fc18.src.rpm
python-psycopg2-2.4.5-5.fc18.src.rpm
python-cement-1.9.14-4.fc18.src.rpm
python-zmq-2.2.0-4.fc18.src.rpm

[edit] Fixes that were needed

[edit] Intermittent failures

I had to apply this fix for a localhost-using test in python-setuptools; this change isn't in git yet:

[david@surprise python-setuptools]$ git diff
diff --git a/python-setuptools.spec b/python-setuptools.spec
index 1dc1a9a..db84cb3 100644
--- a/python-setuptools.spec
+++ b/python-setuptools.spec
@@ -21,7 +21,7 @@ Source2:        zpl.txt
 Patch0: distribute-different-exception-message.patch
 # Sometimes this times out in the build system.  Hanging onto the patch in git
 # for a bit in case that behavior returns
-#Patch1: distribute-timeout-exception.patch
+Patch1: distribute-timeout-exception.patch
 
 BuildRoot:      %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
 
@@ -68,7 +68,7 @@ This package contains the distribute fork of setuptools.
 %setup -q -n %{srcname}-%{version}
 
 %patch0 -p1 -b .excmsg
-#patch1 -p1 -b .exctype
+%patch1 -p1 -b .exctype
 
 find -name '*.txt' | xargs chmod -x
 find . -name '*.orig' -exec rm \{\} \;

[edit] Packaging tweaks

 DONE: python-easygui/python-easygui-0.96-3.fc18.src.rpm
   RPM build errors:
     File not found by glob: /builddir/build/BUILDROOT/python-easygui-0.96-3.fc18.x86_64/usr/lib/python3.3/site-packages/__pycache__/easygui.cpython-32.py*
   Test built in local mock; fix pushed to git master
 DONE: python-smbc/python-smbc-1.0.13-2.fc18.src.rpm
   RPM build errors:
     File not found by glob: /builddir/build/BUILDROOT/python-smbc-1.0.13-2.fc18.x86_64/usr/lib64/python3.3/site-packages/smbc.cpython-3?mu.so
   Test built in local mock; fix pushed to git master
 DONE: python-smbpasswd/python-smbpasswd-1.0.1-19.fc18.src.rpm
    RPM build errors:
      File not found by glob: /builddir/build/BUILDROOT/python-smbpasswd-1.0.1-19.fc18.x86_64/usr/lib64/python3.3/site-packages/smbpasswd.cpython-3?mu.so
   Test built in local mock; fix pushed to git master
 DONE: python-amqplib/python-amqplib-1.0.2-4.fc18.src.rpm
    RPM build errors:
      nosetests-3.2: command not found
    Test built in local mock; fix pushed to git master
 DONE: python-psycopg2/python-psycopg2-2.4.5-2.fc18.src.rpm
     + for python in python python-debug python3 python3.2dmu
     + python3.2dmu setup.py build
     /var/tmp/rpm-tmp.ZyG8xO: line 34: python3.2dmu: command not found
   (also some fileglobs)

[edit] Breaking a circular dep

python-py.spec has:

# we have a circular (build) dependency with the (new) pytest package
# when generating the docs or running the testsuite
%global with_docs 1
%global run_check 1

Disabling both, regening python-py src.rpm and trying again:

[david@surprise python-py]$ git diff
diff --git a/python-py.spec b/python-py.spec
index ef7ad93..8f0eb69 100644
--- a/python-py.spec
+++ b/python-py.spec
@@ -6,8 +6,8 @@
 
 # we have a circular (build) dependency with the (new) pytest package
 # when generating the docs or running the testsuite
-%global with_docs 1
-%global run_check 1
+%global with_docs 0
+%global run_check 0
 
 %global pytest_version 2.2.4

[edit] __import__(, , , level=-1)

In python prior to 3.3. this means "The default is -1 which indicates both absolute and relative imports will be attempted." (c.f. http://docs.python.org/library/functions.html#__import__)

In python 3.3, 0 became the default, meaning to only perform absolute imports and "Changed in version 3.3: Negative values for level are no longer supported." (c.f. http://docs.python.org/dev/library/functions.html#__import__ )

DONE: python-cement/python-cement-1.9.14-2.fc18.src.rpm
  Reported as https://github.com/cement/cement/issues/156; have applied downstream patch

DONE: python-zmq/python-zmq-2.2.0-2.fc18.src.rpm
  File "error.pyx", line 34, in init zmq.core.error (zmq/core/error.c:1195)
     ValueError: level must be >= 0
  .c files generated with old Cython; upon forcing regeneration of .c files it works

[edit] Unicode issues

DONE: numpy/numpy-1.6.2-2.fc18.src.rpm
  Needed porting: has various things that use internals of unicode
  Upstream has ported; we've cherrypicked patches
    http://projects.scipy.org/numpy/ticket/1471
    (also http://projects.scipy.org/numpy/ticket/2145 )
    Discussed here: http://www.mail-archive.com/numpy-discussion@scipy.org/msg38344.html

[edit] Dict ordering issues

Python 3.3 enables hash-randomization by default, making the order of keys in dicts potentially change from run to run.

FIXED: python-ansi2html had an implicit reliance on dict ordering in its selftests.

[edit] Temporary Fixes

[edit] python-nose

I turned off the %check in python-nose in order to build the deps before having the full selftest suite for nose itself passing.

[edit] python-docutils

Tests fail (against 3.3b2): (failures=1, errors=4)

3 failures in test_writers.test_odt.DocutilsOdtTestCase: with "TypeError: expected an Element, not _ElementInterfaceWrapper" odf writer fails when run with Python3.3b1/b2: filed as https://sourceforge.net/tracker/?func=detail&aid=3555164&group_id=38414&atid=422030

Hash-randomization dict ordering issue; sent patch upstream as http://sourceforge.net/tracker/?func=detail&aid=3555160&group_id=38414&atid=422032

Other failures reported as https://sourceforge.net/tracker/?func=detail&aid=3561133&group_id=38414&atid=422030

[edit] Remaining work

[edit] Import issues

[edit] SWIG issues

[edit] TODO: znc/znc-0.206-2.fc18.src.rpm

swig errors

[edit] release notes

added to release notes at https://fedoraproject.org/wiki/Documentation_Development_Beat Immanetize (talk) 17:21, 26 October 2012 (UTC)