From Fedora Project Wiki

Revision as of 13:21, 20 October 2012 by Misc (talk | contribs) (Setup MongoDB, for authentication and datastore)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Wiki version of

so we can edit and adjust


  • complete documentation for node
  • add more test

Setup the Broker

The broker is the component of openshift that will dispatch the creation of each application. It consist of a rails application and can be used with various plugin for serving DNS, storing authentication information and communicating with nodes.

Install OpenShift Origin packages

yum install openshift-origin-broker openshift-origin-cartridge-diy 

Turn off selinux (for now, this step will go away)

setenforce 0
sed -i -e 's|SELINUX=enforcing|SELINUX=permissive|' /etc/sysconfig/selinux
The goal is to have it enabled in the end
So once everything is working, please turn it on again and help us to fix the policy

Setup MongoDB, for authentication and datastore

Openshift Origin use mongodb for storing data and as authentication store. So first, we need to install the plugin

yum install rubygem-openshift-origin-auth-mongo mongodb-server

This part is the same as the original instructions

Make sure the 2 followings configuration are set in /etc/mongodb.conf :

auth = true
smallfiles = true

Save and close the file.

Start mongodb and make sure it starts on reboot

systemctl start  mongod
systemctl enable mongod

Make sure the mongo daemon is running by connecting to it


Sometimes it takes a little while for it to start up. Keep trying until it connects. Then exit out.

Create initial mongodb accounts.

mongo stickshift_broker_dev --eval 'db.addUser("stickshift", "mooo")'
mongo stickshift_broker_dev --eval 'db.auth_user.update({"_id":"admin"}, {"_id":"admin","user":"admin","password":"2a8462d93a13e51387a5e607cbd1139f"}, true)'

Setup the firewall

firewall-cmd --add-service=ssh
firewall-cmd --add-service=https
firewall-cmd --add-service=http

Setup services

chkconfig network on
systemctl enable httpd
systemctl enable openshift-origin-broker
systemctl enable sshd

Setup mcollective, for broker communication

Install the needed rpms for broker communication

yum install rubygem-openshift-origin-msg-broker-mcollective mcollective-qpid-plugin qpid-cpp-server
systemctl enable qpidd
systemctl start qpidd
systemctl enable mcollective

Open the firewall

firewall-cmd --add-port=5672/tcp

Install the needed rpm on the node ( same computer is fine )

yum install openshift-origin-msg-node-mcollective 

Save off /etc/mcollective/client.cfg and /etc/mcollective/server.cfg. Then edit them so that they look like the following. Note: Change to whatever your hostname is. Make also sure that daemmonize is set to 1 (see


topicprefix = /topic/
main_collective = mcollective
collectives = mcollective
libdir = /usr/libexec/mcollective
loglevel = debug
logfile = /var/log/mcollective-client.log

# Plugins
securityprovider = psk
plugin.psk = unset
connector = qpid

# Facts
factsource = yaml
plugin.yaml = /etc/mcollective/facts.yaml


topicprefix = /topic/
main_collective = mcollective
collectives = mcollective
libdir = /usr/libexec/mcollective
logfile = /var/log/mcollective.log
loglevel = debug
daemonize = 1
direct_addressing = n

# Plugins
securityprovider = psk
plugin.psk = unset
connector = qpid

# Facts
factsource = yaml
plugin.yaml = /etc/mcollective/facts.yaml

Setup DNS, based on bind

Install the needed rpms

yum install rubygem-openshift-origin-dns-bind

Note 1: Do the DNS all in one sweep so the variables match

Note 2: Change to whatever your machines domain name is.

Note 3: Yep, except for the first line, you can cut and paste this whole thing.

export keyfile=/var/named/${domain}.key

rm -vf /var/named/K${domain}*
cd /var/named
dnssec-keygen -a HMAC-MD5 -b 512 -n USER -r /dev/urandom ${domain}
KEY="$(grep Key: K${domain}*.private | cut -d ' ' -f 2)"

rndc-confgen -a -r /dev/urandom
restorecon -v /etc/rndc.* /etc/named.*
chown -v root:named /etc/rndc.key
chmod -v 640 /etc/rndc.key

echo "forwarders {;; } ;" > /var/named/forwarders.conf
chmod -v 755 /var/named/forwarders.conf

sed "s/${domain}/g" < /usr/share/gems/gems/openshift-origin-dns-bind-*/doc/examples/ > /var/named/dynamic/${domain}.db

cat > /var/named/${domain}.key <<EOF
key ${domain} {
  algorithm HMAC-MD5;
  secret "${KEY}";
chown -Rv named:named /var/named
restorecon -rv /var/named

mv /etc/named.conf /etc/named.conf.backup
sed "s/${domain}/g" < /usr/share/doc/rubygem-openshift-origin-dns-bind-*/examples/named.conf > /etc/named.conf
chown -v root:named /etc/named.conf
restorecon -v /etc/named.conf

/bin/systemctl start named

Then run "nsupdate -k ${keyfile}" and put in the following Note1: Change to your hostname Note2: Change to your ip address Note3: Type CTR+D (The Control key with the "d" key) to exit out of the program.

update delete A
update add 180 A

Finally, add "nameserver" to the top of /etc/resolv.conf, or add it to Networkmanager configuration.

Open the firewall

firewall-cmd --add-service=dns

Setup Broker plugins

Edit /var/www/stickshift/broker/Gemfile and add the following in the plugin section

gem 'openshift-origin-msg-broker-mcollective'
gem 'openshift-origin-dns-bind'
gem 'openshift-origin-auth-mongo'

Then do the following

cd /var/www/stickshift/broker/
bundle --local

Then edit /var/www/stickshift/broker/config/environments/development.rb and add the following at the very last, after the very last 'end' statement

require File.expand_path('../plugin-config/openshift-origin-msg-broker-mcollective.rb', __FILE__)
require File.expand_path('../plugin-config/openshift-origin-dns-bind.rb', __FILE__)
require File.expand_path('../plugin-config/openshift-origin-auth-mongo.rb', __FILE__)

And then do the following. It is best if you are still in the same session that you did the DNS setup, so that you have the variables still set.

mkdir -p /var/www/stickshift/broker/config/environments/plugin-config
cat > /var/www/stickshift/broker/config/environments/plugin-config/uplift-bind-plugin.rb <EOF
Broker::Application.configure do
  config.dns = {
  :server => "",
  :port => 53,
  :keyname => "${domain}",
  :keyvalue => "${KEY}",
  :zone => "${domain}"

chown -v apache:apache /var/www/stickshift/broker/config/environments/plugin-config/uplift-bind-plugin.rb
restorecon -v /var/www/stickshift/broker/config/environments/plugin-config/uplift-bind-plugin.rb

perl -p -i -e "s/.*:domain_suffix.*/    :domain_suffix =>  \"${domain}\",/" /var/www/stickshift/broker/config/environments/*.rb


In order to make sure everything is fine after a fresh boot, we recommend to reboot.

After your machine has been rebooted, log in, and try the following

curl -Ik https://localhost/broker/rest/api 
  • If you get an error do
curl -k https://localhost/broker/rest/api