Talk:Features/Python 3.3

From FedoraProject

(Difference between revisions)
Jump to: navigation, search
(python-docutils)
(python-docutils)
Line 32: Line 32:
 
</pre>
 
</pre>
  
This one looks like a hash-randomization dict ordering issue:
+
Hash-randomization dict ordering issue; sent patch upstream as http://sourceforge.net/tracker/?func=detail&aid=3555160&group_id=38414&atid=422032
 
<pre>
 
<pre>
 
test_functional.py: functional/tests/standalone_rst_xetex.py; test (test_functional.FunctionalTestCase):
 
test_functional.py: functional/tests/standalone_rst_xetex.py; test (test_functional.FunctionalTestCase):
Line 46: Line 46:
 
  \usepackage{float} % float configuration
 
  \usepackage{float} % float configuration
 
  \floatplacement{figure}{H} % place figures here definitely
 
  \floatplacement{figure}{H} % place figures here definitely
 +
</pre>
 +
 +
<pre>
 +
======================================================================
 +
FAIL: test_parsers/test_rst/test_inline_markup.py: totest['markup recognition rules'][4]; test_parser (DocutilsTestSupport.ParserTestCase)
 +
----------------------------------------------------------------------
 +
Traceback (most recent call last):
 +
  File "/home/david/coding/python3.3/python-docutils/python3-python-docutils-0.10-0.2.20120730svn7490.fc18/test3/DocutilsTestSupport.py", line 459, in test_parser
 +
    self.compare_output(self.input, output, self.expected)
 +
  File "/home/david/coding/python3.3/python-docutils/python3-python-docutils-0.10-0.2.20120730svn7490.fc18/test3/DocutilsTestSupport.py", line 230, in compare_output
 +
    raise error
 +
  File "/home/david/coding/python3.3/python-docutils/python3-python-docutils-0.10-0.2.20120730svn7490.fc18/test3/DocutilsTestSupport.py", line 217, in compare_output
 +
    self.assertEqual(output, expected)
 +
  File "/home/david/coding/python3.3/python-docutils/python3-python-docutils-0.10-0.2.20120730svn7490.fc18/test3/DocutilsTestSupport.py", line 122, in assertEqual
 +
    raise self.failureException(msg or '%s != %s' % _format_str(first, second))
 +
AssertionError: '''\
 +
<document source="test data">
 +
    <paragraph>
 +
        "Quoted" markup start-string (matched openers & closers) -> no markup:
 +
    <paragraph>
 +
        \'*\' "*" (*) <*> [*] {*}
 +
        ⁅*⁆
 +
    <paragraph>
 +
        Some international quoting styles:
 +
        ‘
 +
        <emphasis>
 +
            ’ “
 +
        ” English, ...,
 +
        „*“ ‚*‘ »*« ›
 +
        <emphasis>
 +
            ‹ German, Czech, ...,
 +
            „
 +
        ” «*» Romanian,
 +
        “*„ ‘*‚ Greek,
 +
        「*」 『*』traditional Chinese,
 +
        ”*” ’*’ »*» ›*› Swedish, Finnish,
 +
        „*” ‚*’ Polish,
 +
        „*” »*« ’*’ Hungarian,
 +
    <paragraph>
 +
        But this is „
 +
        <emphasis>
 +
            ’ emphasized »
 +
        ‹.''' != '''\
 +
<document source="test data">
 +
    <paragraph>
 +
        "Quoted" markup start-string (matched openers & closers) -> no markup:
 +
    <paragraph>
 +
        \'*\' "*" (*) <*> [*] {*}
 +
        ⁅*⁆
 +
    <paragraph>
 +
        Some international quoting styles:
 +
        ‘*’ “*” English, ...,
 +
        „*“ ‚*‘ »*« ›*‹ German, Czech, ...,
 +
        „*” «*» Romanian,
 +
        “*„ ‘*‚ Greek,
 +
        「*」 『*』traditional Chinese,
 +
        ”*” ’*’ »*» ›*› Swedish, Finnish,
 +
        „*” ‚*’ Polish,
 +
        „*” »*« ’*’ Hungarian,
 +
    <paragraph>
 +
        But this is „
 +
        <emphasis>
 +
            ’ emphasized »
 +
        ‹.'''
 
</pre>
 
</pre>
  

Revision as of 21:19, 23 August 2012

Contents

Packages that have not yet been rebuilt in koji

I'm working on these. See notes below

Dependency chain of the builds is:

  • python-docutils
    • python-sphinx
      • python-execnet
      • python-manuel
  • scipy
    • python-networkx
  • gcc-python-plugin

python-docutils

Tests fail (against 3.3b2): (failures=1, errors=4)

3 failures in test_writers.test_odt.DocutilsOdtTestCase: with "TypeError: expected an Element, not _ElementInterfaceWrapper" odf writer fails when run with Python3.3b1/b2: filed as https://sourceforge.net/tracker/?func=detail&aid=3555164&group_id=38414&atid=422030

Other failures:

======================================================================
ERROR: test_unicode (test_error_reporting.ErrorStringTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/david/coding/dist-git-fedora/python-docutils/python3-python-docutils-0.10-0.3.20120730svn7490.fc19/test3/test_error_reporting.py", line 154, in test_unicode
    str(ErrorString(ImportError(self.bs))))
  File "/home/david/coding/dist-git-fedora/python-docutils/python3-python-docutils-0.10-0.3.20120730svn7490.fc19/build/lib/docutils/utils/error_reporting.py", line 123, in __str__
    super(ErrorString, self).__str__())
  File "/home/david/coding/dist-git-fedora/python-docutils/python3-python-docutils-0.10-0.3.20120730svn7490.fc19/build/lib/docutils/utils/error_reporting.py", line 73, in __str__
    return str(self.data)
TypeError: __str__ returned non-string (type bytes)

Hash-randomization dict ordering issue; sent patch upstream as http://sourceforge.net/tracker/?func=detail&aid=3555160&group_id=38414&atid=422032

test_functional.py: functional/tests/standalone_rst_xetex.py; test (test_functional.FunctionalTestCase):
--- functional/expected/standalone_rst_xetex.tex
+++ functional/output/standalone_rst_xetex.tex
@@ -7,7 +7,7 @@
 \usepackage{amsmath}
 \usepackage{polyglossia}
 \setdefaultlanguage{english}
-\setotherlanguages{german,british,french}
+\setotherlanguages{german,french,british}
 \usepackage{color}
 \usepackage{float} % float configuration
 \floatplacement{figure}{H} % place figures here definitely
======================================================================
FAIL: test_parsers/test_rst/test_inline_markup.py: totest['markup recognition rules'][4]; test_parser (DocutilsTestSupport.ParserTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/david/coding/python3.3/python-docutils/python3-python-docutils-0.10-0.2.20120730svn7490.fc18/test3/DocutilsTestSupport.py", line 459, in test_parser
    self.compare_output(self.input, output, self.expected)
  File "/home/david/coding/python3.3/python-docutils/python3-python-docutils-0.10-0.2.20120730svn7490.fc18/test3/DocutilsTestSupport.py", line 230, in compare_output
    raise error
  File "/home/david/coding/python3.3/python-docutils/python3-python-docutils-0.10-0.2.20120730svn7490.fc18/test3/DocutilsTestSupport.py", line 217, in compare_output
    self.assertEqual(output, expected)
  File "/home/david/coding/python3.3/python-docutils/python3-python-docutils-0.10-0.2.20120730svn7490.fc18/test3/DocutilsTestSupport.py", line 122, in assertEqual
    raise self.failureException(msg or '%s != %s' % _format_str(first, second))
AssertionError: '''\
<document source="test data">
    <paragraph>
        "Quoted" markup start-string (matched openers & closers) -> no markup:
    <paragraph>
        \'*\' "*" (*) <*> [*] {*}
        ⁅*⁆
    <paragraph>
        Some international quoting styles:
        ‘
        <emphasis>
            ’ “
        ” English, ...,
        „*“ ‚*‘ »*« ›
        <emphasis>
            ‹ German, Czech, ...,
            „
        ” «*» Romanian,
        “*„ ‘*‚ Greek,
        「*」 『*』traditional Chinese,
        ”*” ’*’ »*» ›*› Swedish, Finnish,
        „*” ‚*’ Polish,
        „*” »*« ’*’ Hungarian,
    <paragraph>
        But this is „
        <emphasis>
            ’ emphasized »
        ‹.''' != '''\
<document source="test data">
    <paragraph>
        "Quoted" markup start-string (matched openers & closers) -> no markup:
    <paragraph>
        \'*\' "*" (*) <*> [*] {*}
        ⁅*⁆
    <paragraph>
        Some international quoting styles:
        ‘*’ “*” English, ...,
        „*“ ‚*‘ »*« ›*‹ German, Czech, ...,
        „*” «*» Romanian,
        “*„ ‘*‚ Greek,
        「*」 『*』traditional Chinese,
        ”*” ’*’ »*» ›*› Swedish, Finnish,
        „*” ‚*’ Polish,
        „*” »*« ’*’ Hungarian,
    <paragraph>
        But this is „
        <emphasis>
            ’ emphasized »
        ‹.'''

python-manuel

BR: python3-sphinx

python-nose1.1

selftest failures: FAILED (SKIP=9, errors=1, failures=3)

python-sphinx

BR: python3-docutils

python-execnet

BR: python3-sphinx

scipy

Cython sources need regenerating

  File "orthogonal_eval.pyx", line 93, in init scipy.special.orthogonal_eval (scipy/special/orthogonal_eval.c:4126)
ValueError: level must be >= 0

(also some swig issues, I think)


python-networkx

BR: python3-scipy

gcc-python-plugin

(in progress)

Packages that have been rebuilt in koji

python3
python-setuptools
deltarpm
dreampie
lcgdm
libselinux
libsemanage
loook
metagoofil
mpi4py
mysql-connector-python
nordugrid-arc
pygpgme
pyke
pyparsing
python3-cairo
python3-chardet
python3-postgresql
python-asciitable
python-auth-credential
python-cmd2
python-crypto
python-dirq
python-httplib2
python-inotify
python-messaging
python-mock
python-pbs
python-ply
python-psutil
python-requests
python-restauth-common
python-restauth
python-six
python-sleekxmpp
python-straight-plugin
revisor
ris-linux
sip
stomppy
uwsgi
waf
pygobject3
PyQt4
ATpy
babel
bpython
nfoview
pyshp
python3-cherrypy
python3-dateutil
python3-zope-fixers
python-anyjson
python-coverage
python-cvxopt
python-decorator
python-enchant
python-feedparser
python-flexmock
python-html5lib
python-lxml
python-markupsafe
python-minimock
python-nose-cover3
python-pip
python-pkgwat-api
python-profilehooks
python-pyro
python-simplegeneric
python-tempita
python-tornado
python-urwid
python-zope-event
PyYAML
python-beautifulsoup4
python-BeautifulSoup
python-jinja2
python-easygui
python-smbc
python-smbpasswd
python-py
pycmd
autoarchive
python-psycopg2
python-mglob
python-nose
python-beaker
python-markdown
python-pygments
python-amqplib
python-cement
python-zmq
python-mako
python-markdown2
pytest
numpy
pyfits
pyp2rpm
python-ansi2html
python-setproctitle
python-nose1.1


Scope of python3 in Fedora 18

Getting python3-using packages:

$ for f in $(repoquery --whatrequires python3) ; do echo $f ; repoquery --source $f ; done > sources.txt
$ grep src.rpm sources.txt | sort | uniq | wc -l
130

but it's showing some duplicates; adding filtering shows 105 src.rpms:

$ grep src.rpm sources.txt | python -c "import sys, re; print('\n'.join([re.match('(.*)-(.*)-(.*).src.rpm', line).group(1) for line in sys.stdin.readlines()]))"| sort |uniq
ATpy
babel
bpython
cobbler
deltarpm
dreampie
lcgdm
libselinux
libsemanage
loook
metagoofil
mpi4py
mysql-connector-python
nfoview
nordugrid-arc
numpy
pycmd
pyfits
pygobject3
pyke
pyp2rpm
pyparsing
PyQt4
pyshp
pytest
python3
python3-cairo
python3-chardet
python3-cherrypy
python3-dateutil
python3-postgresql
python3-zope-fixers
python-amqplib
python-ansi2html
python-anyjson
python-asciitable
python-auth-credential
python-beaker
python-BeautifulSoup
python-beautifulsoup4
python-cmd2
python-coverage
python-crypto
python-cvxopt
python-decorator
python-dirq
python-docutils
python-easygui
python-enchant
python-execnet
python-feedparser
python-flexmock
python-html5lib
python-httplib2
python-inotify
python-jinja2
python-lxml
python-mako
python-manuel
python-markdown
python-markdown2
python-markupsafe
python-messaging
python-mglob
python-minimock
python-mock
python-networkx
python-nose
python-nose1.1
python-pbs
python-pip
python-pkgwat-api
python-ply
python-psutil
python-psycopg2
python-py
python-pygments
python-pyro
python-requests
python-restauth
python-restauth-common
python-setproctitle
python-setuptools
python-simplegeneric
python-six
python-sleekxmpp
python-smbc
python-smbpasswd
python-sphinx
python-sqlalchemy
python-straight-plugin
python-tempita
python-tornado
python-urwid
python-zmq
python-zope-event
PyYAML
revisor
ris-linux
scipy
sip
stomppy
uwsgi
waf
znc

What rebuilds cleanly

After building "python3" from the "python3.3" branch, mockchain built the following successfully in order: (using this script to obtain the build order):

[david@surprise python3.3]$ grep Success localrepo/mockchain*.log |python -c"import sys, re; print '\n'.join([re.match('.*Success building (.+)', line).group(1) for line in sys.stdin.readlines()])"
cobbler-2.2.3-3.fc18.src.rpm
deltarpm-3.6-0.9.20110223git.fc18.src.rpm
dreampie-1.1.1-2.fc18.src.rpm
lcgdm-1.8.3.1-4.fc18.src.rpm
libselinux-2.1.11-4.fc18.src.rpm
libsemanage-2.1.8-4.fc18.src.rpm
loook-0.6.7-8.fc18.src.rpm
metagoofil-1.4b-5.fc18.src.rpm
mpi4py-1.3-3.fc18.src.rpm
mysql-connector-python-0.3.2-4.fc18.src.rpm
nordugrid-arc-2.0.0-3.fc18.src.rpm
pygpgme-0.3-2.fc18.src.rpm
pyke-1.1.1-8.fc18.src.rpm
pyparsing-1.5.6-4.fc18.src.rpm
python3-cairo-1.10.0-3.fc18.src.rpm
python3-chardet-2.0.1-7.fc18.src.rpm
python3-postgresql-1.0.2-3.fc18.src.rpm
python-asciitable-0.8.0-3.fc18.src.rpm
python-auth-credential-0.6-2.fc18.src.rpm
python-cmd2-0.6.4-3.fc18.src.rpm
python-crypto-2.6-2.fc18.src.rpm
python-dirq-1.2.2-2.fc18.src.rpm
python-httplib2-0.7.4-5.fc18.src.rpm
python-inotify-0.9.3-2.fc18.src.rpm
python-messaging-0.10-2.fc18.src.rpm
python-mock-0.8.0-3.fc18.src.rpm
python-pbs-4.3.3-2.fc18.src.rpm
python-ply-3.4-2.fc18.src.rpm
python-psutil-0.5.1-2.fc18.src.rpm
python-requests-0.13.1-2.fc18.src.rpm
python-restauth-common-0.5.1-5.fc18.src.rpm
python-restauth-0.5.1-5.fc18.src.rpm
python-six-1.1.0-3.fc18.src.rpm
python-sleekxmpp-1.0-0.10.beta2.fc18.src.rpm
python-straight-plugin-1.1.1-0.9.20111110.git57ef11c.fc18.src.rpm
revisor-2.2-5.fc18.src.rpm
ris-linux-0.4-10.fc18.src.rpm
sip-4.13.3-2.fc18.src.rpm
stomppy-3.0.5-2.fc18.src.rpm
uwsgi-1.2.4-2.fc18.src.rpm
waf-1.7.0-1.fc18.1.src.rpm
pygobject3-3.3.4-1.fc18.src.rpm
PyQt4-4.9.4-2.fc18.src.rpm
python-setuptools-0.6.28-1.fc18.src.rpm
ATpy-0.9.6-2.fc18.src.rpm
babel-0.9.6-3.fc18.src.rpm
bpython-0.11-2.fc18.src.rpm
nfoview-1.10-4.fc18.src.rpm
pyshp-1.1.4-4.fc18.src.rpm
python3-cherrypy-3.2.0-0.rc1.r2567.1.fc18.5.src.rpm
python3-dateutil-2.0-3.fc18.src.rpm
python3-zope-fixers-1.0-3.fc18.src.rpm
python-anyjson-0.3.1-4.fc18.src.rpm
python-coverage-3.5.2-0.2.b1.fc18.src.rpm
python-cvxopt-1.1.5-3.fc18.src.rpm
python-decorator-3.3.3-2.fc18.src.rpm
python-enchant-1.6.5-6.fc18.src.rpm
python-feedparser-5.1.2-3.fc18.src.rpm
python-flexmock-0.9.4-2.fc18.src.rpm
python-html5lib-0.95-2.fc18.src.rpm
python-lxml-2.3.3-2.fc18.src.rpm
python-markupsafe-0.11-5.fc18.src.rpm
python-minimock-1.2.7-3.fc18.src.rpm
python-nose-cover3-0.1.0-3.fc18.src.rpm
python-pip-1.1-2.fc18.src.rpm
python-pkgwat-api-0.3-4.fc18.src.rpm
python-profilehooks-1.6-2.fcf18.src.rpm
python-pyro-4.9-3.fc18.src.rpm
python-simplegeneric-0.8-4.fc18.src.rpm
python-tempita-0.5.1-2.fc18.src.rpm
python-tornado-2.2.1-2.fc18.src.rpm
python-urwid-1.0.0-4.fc18.src.rpm
python-zope-event-3.5.1-3.fc18.src.rpm
PyYAML-3.10-4.fc18.src.rpm
python-beautifulsoup4-4.1.1-4.fc18.src.rpm
python-BeautifulSoup-3.2.1-2.fc18.src.rpm
python-jinja2-2.6-3.fc18.src.rpm
python-easygui-0.96-4.fc18.src.rpm
python-smbc-1.0.13-3.fc18.src.rpm
python-smbpasswd-1.0.1-20.fc18.src.rpm
python-py-1.4.9-2.fc18.src.rpm
pycmd-1.0-6.fc18.src.rpm
pytest-2.2.4-2.fc18.src.rpm
python-execnet-1.1-2.fc18.src.rpm
python-nose-1.1.2-3.fc18.src.rpm
python-ansi2html-0.9.1-6.fc18.src.rpm
python-beaker-1.5.4-4.fc18.src.rpm
python-mako-0.5.0-3.fc18.src.rpm
python-markdown-2.2.0-2.fc18.src.rpm
python-pygments-1.4-5.fc18.src.rpm
python-markdown2-2.0.0-2.fc18.src.rpm
python-amqplib-1.0.2-5.fc18.src.rpm
python-psycopg2-2.4.5-5.fc18.src.rpm
python-cement-1.9.14-4.fc18.src.rpm
python-zmq-2.2.0-4.fc18.src.rpm

Fixes that were needed

Intermittent failures

I had to apply this fix for a localhost-using test in python-setuptools; this change isn't in git yet:

[david@surprise python-setuptools]$ git diff
diff --git a/python-setuptools.spec b/python-setuptools.spec
index 1dc1a9a..db84cb3 100644
--- a/python-setuptools.spec
+++ b/python-setuptools.spec
@@ -21,7 +21,7 @@ Source2:        zpl.txt
 Patch0: distribute-different-exception-message.patch
 # Sometimes this times out in the build system.  Hanging onto the patch in git
 # for a bit in case that behavior returns
-#Patch1: distribute-timeout-exception.patch
+Patch1: distribute-timeout-exception.patch
 
 BuildRoot:      %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
 
@@ -68,7 +68,7 @@ This package contains the distribute fork of setuptools.
 %setup -q -n %{srcname}-%{version}
 
 %patch0 -p1 -b .excmsg
-#patch1 -p1 -b .exctype
+%patch1 -p1 -b .exctype
 
 find -name '*.txt' | xargs chmod -x
 find . -name '*.orig' -exec rm \{\} \;

Packaging tweaks

 DONE: python-easygui/python-easygui-0.96-3.fc18.src.rpm
   RPM build errors:
     File not found by glob: /builddir/build/BUILDROOT/python-easygui-0.96-3.fc18.x86_64/usr/lib/python3.3/site-packages/__pycache__/easygui.cpython-32.py*
   Test built in local mock; fix pushed to git master
 DONE: python-smbc/python-smbc-1.0.13-2.fc18.src.rpm
   RPM build errors:
     File not found by glob: /builddir/build/BUILDROOT/python-smbc-1.0.13-2.fc18.x86_64/usr/lib64/python3.3/site-packages/smbc.cpython-3?mu.so
   Test built in local mock; fix pushed to git master
 DONE: python-smbpasswd/python-smbpasswd-1.0.1-19.fc18.src.rpm
    RPM build errors:
      File not found by glob: /builddir/build/BUILDROOT/python-smbpasswd-1.0.1-19.fc18.x86_64/usr/lib64/python3.3/site-packages/smbpasswd.cpython-3?mu.so
   Test built in local mock; fix pushed to git master
 DONE: python-amqplib/python-amqplib-1.0.2-4.fc18.src.rpm
    RPM build errors:
      nosetests-3.2: command not found
    Test built in local mock; fix pushed to git master
 DONE: python-psycopg2/python-psycopg2-2.4.5-2.fc18.src.rpm
     + for python in python python-debug python3 python3.2dmu
     + python3.2dmu setup.py build
     /var/tmp/rpm-tmp.ZyG8xO: line 34: python3.2dmu: command not found
   (also some fileglobs)

Breaking a circular dep

python-py.spec has:

# we have a circular (build) dependency with the (new) pytest package
# when generating the docs or running the testsuite
%global with_docs 1
%global run_check 1

Disabling both, regening python-py src.rpm and trying again:

[david@surprise python-py]$ git diff
diff --git a/python-py.spec b/python-py.spec
index ef7ad93..8f0eb69 100644
--- a/python-py.spec
+++ b/python-py.spec
@@ -6,8 +6,8 @@
 
 # we have a circular (build) dependency with the (new) pytest package
 # when generating the docs or running the testsuite
-%global with_docs 1
-%global run_check 1
+%global with_docs 0
+%global run_check 0
 
 %global pytest_version 2.2.4

__import__(, , , level=-1)

In python prior to 3.3. this means "The default is -1 which indicates both absolute and relative imports will be attempted." (c.f. http://docs.python.org/library/functions.html#__import__)

In python 3.3, 0 became the default, meaning to only perform absolute imports and "Changed in version 3.3: Negative values for level are no longer supported." (c.f. http://docs.python.org/dev/library/functions.html#__import__ )

DONE: python-cement/python-cement-1.9.14-2.fc18.src.rpm
  Reported as https://github.com/cement/cement/issues/156; have applied downstream patch

DONE: python-zmq/python-zmq-2.2.0-2.fc18.src.rpm
  File "error.pyx", line 34, in init zmq.core.error (zmq/core/error.c:1195)
     ValueError: level must be >= 0
  .c files generated with old Cython; upon forcing regeneration of .c files it works

Unicode issues

DONE: numpy/numpy-1.6.2-2.fc18.src.rpm
  Needed porting: has various things that use internals of unicode
  Upstream has ported; we've cherrypicked patches
    http://projects.scipy.org/numpy/ticket/1471
    (also http://projects.scipy.org/numpy/ticket/2145 )
    Discussed here: http://www.mail-archive.com/numpy-discussion@scipy.org/msg38344.html

Dict ordering issues

Python 3.3 enables hash-randomization by default, making the order of keys in dicts potentially change from run to run.

FIXED: python-ansi2html had an implicit reliance on dict ordering in its selftests.

Temporary Fixes

python-nose

I turned off the %check in python-nose in order to build the deps before having the full selftest suite for nose itself passing.

Remaining work

Import issues

SWIG issues

TODO: znc/znc-0.206-2.fc18.src.rpm

swig errors