User:Hamzy/CloudSIG ppc64le issues

CentOS Cloud SIG ppc64le issues

The following is a list of open issues needed to get https://buildlogs.centos.org/centos/7/cloud/ppc64le/openstack-pike/ matching https://buildlogs.centos.org/centos/7/cloud/x86_64/openstack-pike/

Taging new versions into cloud7-openstack-common-release

libunwind

libunwind-1.1-3.el7.src.rpm does not support ppc64le architecture

...
error: Architecture is not included: ppc64le
...

We need to tag libunwind-1.1-10.el7 into cloud7-openstack-common-release

Merging ppc64le scratch build into existing build

All of the packages have been processed.

For history's sake, the following packages needed to have scratch builds for only ppc64le built and then merged into the existing build of that version (with breaks for repo regens):

cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 apiextractor-0.10.10-11.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 python-dulwich-0.16.1-1.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 fio-2.2.10-1.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 jsoncpp-0.10.5-2.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 librdkafka-0.9.2-2.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 openvswitch-2.7.1-1.1fc27.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 Cython-0.25.2-3.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 python-dulwich-0.16.1-1.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 python-lz4-0.9.0-1.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 python-psutil-5.2.2-1.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 python-tornado-4.4.2-2.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 python-gitdb-0.6.4-4.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 qpid-proton-0.14.0-2.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 sip-4.19-1.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 sparsehash-2.0.2-4.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 mozjs38-38.8.0-2.el7.src.rpm
---
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 coin-or-CoinUtils-2.10.13-1.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 generatorrunner-0.6.16-10.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 mongodb-3.2.8-3.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 libev-4.24-1.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 qpid-cpp-1.35.0-2.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 python-subprocess32-3.2.6-4.el7.src.rpm
---
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 glpk-4.52.1-2.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 PyQt4-4.12-1.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 python-cassandra-driver-3.7.1-4.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 python-gevent-1.1.2-2.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 python-matplotlib-2.0.0-2.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 shiboken-1.2.4-2.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 cmake3-3.6.1-2.el7.src.rpm
---
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 coin-or-Osi-0.107.8-1.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 python-pandas-0.19.1-2.el7.2.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 python-pykafka-2.6.0-0.1.dev2.el7.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 python-pyside-1.2.2-6.el7.1.src.rpm
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 qpid-dispatch-0.6.1-4.el7.src.rpm
---
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 coin-or-Clp-1.16.10-1.el7.src.rpm
---
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 coin-or-Cgl-0.59.9-1.el7.src.rpm
---
cbs build --scratch --arch-override=ppc64le cloud7-openstack-common-el7 coin-or-Cbc-2.9.8-1.el7.src.rpm

Needs to move from cloud7-openstack-common-candidate to cloud7-openstack-common-release

How in general is this supposed to happen? Also, specifically, these packages need to show up in -release.

corosync-2.4.0-9.el7
https://cbs.centos.org/koji/buildinfo?buildID=17261
erlang-19.3.6.1-1.el7
https://cbs.centos.org/koji/buildinfo?buildID=17627
galera-25.3.16-3.el7
https://cbs.centos.org/koji/buildinfo?buildID=17566
libqb-1.0.1-3.el7
https://cbs.centos.org/koji/buildinfo?buildID=17248
librx-1.5-28.el7
https://cbs.centos.org/koji/buildinfo?buildID=16943
nagios-4.0.8-2.el7
https://cbs.centos.org/koji/buildinfo?buildID=11208
pacemaker-1.1.16-10.el7
https://cbs.centos.org/koji/buildinfo?buildID=17258
pcs-0.9.158-4.el7
https://cbs.centos.org/koji/buildinfo?buildID=17327
python-bcrypt-3.1.2-3.el7
https://cbs.centos.org/koji/buildinfo?buildID=17369
python-pillow-4.0.0-0.1.el7
https://cbs.centos.org/koji/buildinfo?buildID=16917
scipy
https://cbs.centos.org/koji/buildinfo?buildID=15956
python-sqlalchemy
https://cbs.centos.org/koji/buildinfo?buildID=17555
resource-agents
https://cbs.centos.org/koji/buildinfo?buildID=17263
subunit
https://cbs.centos.org/koji/buildinfo?buildID=17618

cloud7-openstack-pike-candidate is not built yet for ppc64le

The repository https://cbs.centos.org/repos/cloud7-openstack-pike-candidate/ is not currently built for ppc64 because it would break the x86_64 builds.

Other package errors

python-tables

Building python-tables-3.1.1-2.el7.src.rpm encounters the following error:

+ python setup.py build
* Using Python 2.7.5 (default, Nov 11 2016, 04:49:48) 
.. ERROR:: You need numpy 1.4.1 or greater to run PyTables!
RPM build errors:
error: Bad exit status from /var/tmp/rpm-tmp.U2vx9n (%build)
    Bad exit status from /var/tmp/rpm-tmp.U2vx9n (%build)
Child return code was: 1

x86_64 successfully built a while ago, but a scratch build also fails now.

Building python-tables-3.2.0-1.el7.src.rpm encounters the following error:

...
DEBUG util.py:417:  Error: No Package found for blosc-devel >= 1.5.2
...

So blosc-1.6.1-1.el7.src.rpm was imported from EPEL. However, python-tables still fails (https://kojipkgs.fedoraproject.org//packages/python-tables/3.2.0/1.el7/data/logs/ppc64le/build.log):

...
ERROR: None (tables.tests.test_tables.RecArrayRangeTestCase)
...
ValueError: could not broadcast input array from shape (2) into shape (0)
...
Ran 5734 tests in 183.285s
FAILED (errors=4, skipped=13)
...

Maybe https://github.com/PyTables/PyTables/issues/482 points to a solution?

Source package does not support ppc64le architecture

mariadb-galera

mariadb-galera-5.5.40-3.el7.src.rpm

openstack-tripleo-ui-deps/phantomjs

openstack-tripleo-ui-deps-7-5.el7.src.rpm phantomjs-1.9.7-3.el7.src.rpm

python-cradox

python-cradox-1.3.2-1.el7.src.rpm

v8

v8-3.14.5.10-17.el7.src.rpm

DLRN builds

dlrn has the following modifications:

diff --git a/projects.ini b/projects.ini
index c9f165d..a9b602d 100644
--- a/projects.ini
+++ b/projects.ini
@@ -1,7 +1,8 @@
 [DEFAULT]
 datadir=./data
 scriptsdir=./scripts
-baseurl=http://trunk.rdoproject.org/centos7/
+#baseurl=http://trunk.rdoproject.org/centos7/
+baseurl=http://127.0.0.1/openstack-pike/
 distro=rpm-master
 source=master
 target=centos
@@ -10,7 +11,7 @@ reponame=delorean
 templatedir=./dlrn/templates
 maxretries=3
 pkginfo_driver=dlrn.drivers.rdoinfo.RdoInfoDriver
-tags=
+tags=pike-uc
 #tags=mitaka
 rsyncdest=
 rsyncport=22
diff --git a/scripts/centos.cfg b/scripts/centos.cfg
index 9298f1a..7dc8d99 100644
--- a/scripts/centos.cfg
+++ b/scripts/centos.cfg
@@ -1,11 +1,24 @@
-config_opts['root'] = 'dlrn-centos7-x86_64'
-config_opts['target_arch'] = 'x86_64'
-config_opts['legal_host_arches'] = ('x86_64',)
+config_opts['root'] = 'dlrn-centos7-ppc64le'
+config_opts['target_arch'] = 'ppc64le'
+config_opts['legal_host_arches'] = ('ppc64le',)
 config_opts['chroot_setup_cmd'] = 'install basesystem rpm-build python2-devel gcc make python-sqlalchemy python-webob ghostscript g
 config_opts['dist'] = 'el7'  # only useful for --resultdir variable subst
 config_opts['releasever'] = '7'
 config_opts['plugin_conf']['ccache_enable'] = False
 config_opts['priorities.conf'] = """
+config_opts['nosync'] = True
+config_opts['plugin_conf']['root_cache_opts']['compress_program'] = ""
+config_opts['plugin_conf']['root_cache_opts']['extension'] = ""
+config_opts['plugin_conf']['tmpfs_enable'] = True
+config_opts['plugin_conf']['tmpfs_opts'] = {}
+config_opts['plugin_conf']['tmpfs_opts']['required_ram_mb'] = 1024
+config_opts['plugin_conf']['tmpfs_opts']['max_fs_size'] = '50g'
+config_opts['plugin_conf']['tmpfs_opts']['mode'] = '0755'
+config_opts['plugin_conf']['tmpfs_opts']['keep_mounted'] = False
+config_opts['macros']['%__gzip'] = '/usr/bin/pigz'
+config_opts['macros']['%__bzip2'] = '/usr/bin/lbzip2'
+config_opts['http_proxy'] = 'http://127.0.0.1:3128'
+
 [main]
 enabled = 1
 check_obsoletes = 1
@@ -29,23 +42,30 @@ syslog_device=
 
 [base]
 name=BaseOS
-mirrorlist=http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=os
+#mirrorlist=http://mirrorlist.centos.org/?release=7&arch=ppc64le&repo=os
+baseurl=http://mirror.centos.org/altarch/$releasever/os/$basearch/
 failovermethod=priority
 gpgkey=file:///etc/pki/mock/RPM-GPG-KEY-CentOS-7
+       file:///etc/pki/mock/RPM-GPG-KEY-CentOS-SIG-AltArch-7-$basearch
 gpgcheck=1
 
 [updates]
 name=updates
 enabled=1
-mirrorlist=http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=updates
+#mirrorlist=http://mirrorlist.centos.org/?release=7&arch=ppc64le&repo=updates
+baseurl=http://mirror.centos.org/altarch/$releasever/updates/$basearch/
 failovermethod=priority
 gpgkey=file:///etc/pki/mock/RPM-GPG-KEY-CentOS-7
+       file:///etc/pki/mock/RPM-GPG-KEY-CentOS-SIG-AltArch-7-$basearch
 gpgcheck=1
 
 [extras]
 name=extras
-mirrorlist=http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=extras
+#mirrorlist=http://mirrorlist.centos.org/?release=7&arch=ppc64le&repo=extras
+baseurl=http://mirror.centos.org/altarch/$releasever/extras/$basearch/
 failovermethod=priority
 gpgkey=file:///etc/pki/mock/RPM-GPG-KEY-EPEL-7
+       file:///etc/pki/mock/RPM-GPG-KEY-CentOS-SIG-AltArch-7-$basearch
 gpgcheck=1
+exclude=python-greenlet
 """

[hamzy@pkvmci853 DLRN]$ cat << __EOF__ | sudo tee -a /var/www/html/openstack-pike/delorean-deps.repo
[cloud7_openstack_common_candidate]
name=cloud7_openstack_common_candidate
baseurl=https://cbs.centos.org/repos/cloud7-openstack-common-candidate/ppc64le/os/
enabled=1
gpgcheck=0
cost=1

[cloud7_openstack_common_testing]
name=cloud7_openstack_common_testing
baseurl=https://cbs.centos.org/repos/cloud7-openstack-common-testing/ppc64le/os/
enabled=1
gpgcheck=0
cost=2

[cloud7_openstack_common_release]
name=cloud7_openstack_common_release
baseurl=https://cbs.centos.org/repos/cloud7-openstack-common-release/ppc64le/os/
enabled=1
gpgcheck=0
cost=3

[openstack_pike_ppc64le]
name=openstack_pike_ppc64le
baseurl=https://buildlogs.centos.org/centos/7/cloud/ppc64le/openstack-pike/
enabled=1
gpgcheck=0
cost=4

[openstack_pike_x8664]
name=openstack_pike_x8664
baseurl=https://buildlogs.centos.org/centos/7/cloud/x86_64/openstack-pike/
enabled=1
gpgcheck=0
cost=5

When dlrn builds a clean set of packages:

[hamzy@pkvmci853 ~]$ source ~/dlrn-venv/bin/activate
(dlrn-venv)[hamzy@pkvmci853 DLRN]$ rm -rf data/; /bin/rm commits.sqlite; dlrn --order --verbose-mock 2>&1 | tee output.dlrn

missing dlrn order packages

there are a couple of missing packages that are in cloud7-openstack-pike-xxx but are not built by dlrn:

python-psutil
openvswitch

python-openstackclient-3.11.0-0.20170720225953.c69304e

python-openstackclient-3.11.0-0.20170720225953.c69304e.el7.centos.src.rpm (https://github.com/openstack/python-openstackclient/commit/c69304e3d365dc2c67fab298eba0b9097d3819da)

has the following error during the build tests:

...
2017-07-20 19:02:22,741 INFO:dlrn-build:DEBUG: FAIL: openstackclient.tests.unit.integ.cli.test_shell.TestIntegShellCliPrecedenceOCC.test_shell_args_precedence_1
...
2017-07-20 19:02:22,748 INFO:dlrn-build:DEBUG: testtools.matchers._impl.MismatchError: 0 != 2
...

I believe that this problem has been fixed by the following defects:

- https://bugs.launchpad.net/os-client-config/+bug/1694937

 - https://trunk.rdoproject.org/centos7-master-head/d0/34/d034b980ab03bbd50d31ef8bd439bc80a91a7d77_b81ca2c8/rpmbuild.log

- https://bugs.launchpad.net/python-openstackclient/+bug/1703783