From Fedora Project Wiki

Wiki version of https://openshift.redhat.com/community/forums/openshift/fedora-18-openshift-origin-setup-steps-and-testing

so we can edit and adjust

TODO

  • complete documentation for node
  • add more test

Setup the Broker

The broker is the component of openshift that will dispatch the creation of each application. It consist of a rails application and can be used with various plugin for serving DNS, storing authentication information and communicating with nodes.

Install OpenShift Origin packages

yum install openshift-origin-broker openshift-origin-cartridge-diy 

Turn off selinux (for now, this step will go away)

setenforce 0
sed -i -e 's|SELINUX=enforcing|SELINUX=permissive|' /etc/sysconfig/selinux
The goal is to have it enabled in the end
So once everything is working, please turn it on again and help us to fix the policy


Setup MongoDB, for authentication and datastore

Openshift Origin use mongodb for storing data and as authentication store. So first, we need to install the plugin

yum install rubygem-openshift-origin-auth-mongo mongodb-server

This part is the same as the original instructions

Make sure the 2 followings configuration are set in /etc/mongodb.conf :

auth = true
smallfiles = true

Save and close the file.

Start mongodb and make sure it starts on reboot

systemctl start  mongod
systemctl enable mongod

Make sure the mongo daemon is running by connecting to it

mongo

Sometimes it takes a little while for it to start up. Keep trying until it connects. Then exit out.

Create initial mongodb accounts.

mongo stickshift_broker_dev --eval 'db.addUser("stickshift", "mooo")'
mongo stickshift_broker_dev --eval 'db.auth_user.update({"_id":"admin"}, {"_id":"admin","user":"admin","password":"2a8462d93a13e51387a5e607cbd1139f"}, true)'

Setup the firewall

firewall-cmd --add-service=ssh
firewall-cmd --add-service=https
firewall-cmd --add-service=http

Setup services

chkconfig network on
systemctl enable httpd
systemctl enable openshift-origin-broker
systemctl enable sshd

Setup mcollective, for broker communication

Install the needed rpms for broker communication

yum install rubygem-openshift-origin-msg-broker-mcollective mcollective-qpid-plugin qpid-cpp-server
systemctl enable qpidd
systemctl start qpidd
systemctl enable mcollective

Open the firewall

firewall-cmd --add-port=5672/tcp

Install the needed rpm on the node ( same computer is fine )

yum install openshift-origin-msg-node-mcollective 

Save off /etc/mcollective/client.cfg and /etc/mcollective/server.cfg. Then edit them so that they look like the following. Note: Change broker.example.com to whatever your hostname is. Make also sure that daemmonize is set to 1 (see https://bugzilla.redhat.com/show_bug.cgi?id=868417)

/etc/mcollective/client.cfg

topicprefix = /topic/
main_collective = mcollective
collectives = mcollective
libdir = /usr/libexec/mcollective
loglevel = debug
logfile = /var/log/mcollective-client.log

# Plugins
securityprovider = psk
plugin.psk = unset
connector = qpid
plugin.qpid.host=broker.example.com
plugin.qpid.secure=false
plugin.qpid.timeout=5

# Facts
factsource = yaml
plugin.yaml = /etc/mcollective/facts.yaml

/etc/mcollective/server.cfg


topicprefix = /topic/
main_collective = mcollective
collectives = mcollective
libdir = /usr/libexec/mcollective
logfile = /var/log/mcollective.log
loglevel = debug
daemonize = 1
direct_addressing = n

# Plugins
securityprovider = psk
plugin.psk = unset
connector = qpid
plugin.qpid.host=broker.example.com
plugin.qpid.secure=false
plugin.qpid.timeout=5

# Facts
factsource = yaml
plugin.yaml = /etc/mcollective/facts.yaml

Setup DNS, based on bind

Install the needed rpms

yum install rubygem-openshift-origin-dns-bind

Note 1: Do the DNS all in one sweep so the variables match

Note 2: Change example.com to whatever your machines domain name is.

Note 3: Yep, except for the first line, you can cut and paste this whole thing.

export domain=example.com
export keyfile=/var/named/${domain}.key

rm -vf /var/named/K${domain}*
cd /var/named
dnssec-keygen -a HMAC-MD5 -b 512 -n USER -r /dev/urandom ${domain}
KEY="$(grep Key: K${domain}*.private | cut -d ' ' -f 2)"

rndc-confgen -a -r /dev/urandom
restorecon -v /etc/rndc.* /etc/named.*
chown -v root:named /etc/rndc.key
chmod -v 640 /etc/rndc.key

echo "forwarders { 8.8.8.8; 8.8.4.4; } ;" > /var/named/forwarders.conf
chmod -v 755 /var/named/forwarders.conf

sed "s/example.com/${domain}/g" < /usr/share/gems/gems/openshift-origin-dns-bind-*/doc/examples/example.com.db > /var/named/dynamic/${domain}.db

cat > /var/named/${domain}.key <<EOF
key ${domain} {
  algorithm HMAC-MD5;
  secret "${KEY}";
};
EOF
chown -Rv named:named /var/named
restorecon -rv /var/named

mv /etc/named.conf /etc/named.conf.backup
sed "s/example.com/${domain}/g" < /usr/share/doc/rubygem-openshift-origin-dns-bind-*/examples/named.conf > /etc/named.conf
chown -v root:named /etc/named.conf
restorecon -v /etc/named.conf

/bin/systemctl start named

Then run "nsupdate -k ${keyfile}" and put in the following Note1: Change broker.example.com to your hostname Note2: Change 10.0.0.1 to your ip address Note3: Type CTR+D (The Control key with the "d" key) to exit out of the program.

server 127.0.0.1
update delete broker.example.com A
update add broker.example.com 180 A 10.0.0.1
send

Finally, add "nameserver 127.0.0.1" to the top of /etc/resolv.conf, or add it to Networkmanager configuration.

Open the firewall

firewall-cmd --add-service=dns

Setup Broker plugins

Edit /var/www/stickshift/broker/Gemfile and add the following in the plugin section

gem 'openshift-origin-msg-broker-mcollective'
gem 'openshift-origin-dns-bind'
gem 'openshift-origin-auth-mongo'

Then do the following

cd /var/www/stickshift/broker/
bundle --local

Then edit /var/www/stickshift/broker/config/environments/development.rb and add the following at the very last, after the very last 'end' statement

require File.expand_path('../plugin-config/openshift-origin-msg-broker-mcollective.rb', __FILE__)
require File.expand_path('../plugin-config/openshift-origin-dns-bind.rb', __FILE__)
require File.expand_path('../plugin-config/openshift-origin-auth-mongo.rb', __FILE__)

And then do the following. It is best if you are still in the same session that you did the DNS setup, so that you have the variables still set.

mkdir -p /var/www/stickshift/broker/config/environments/plugin-config
cat > /var/www/stickshift/broker/config/environments/plugin-config/uplift-bind-plugin.rb <EOF
Broker::Application.configure do
  config.dns = {
  :server => "127.0.0.1",
  :port => 53,
  :keyname => "${domain}",
  :keyvalue => "${KEY}",
  :zone => "${domain}"
}
end
EOF

chown -v apache:apache /var/www/stickshift/broker/config/environments/plugin-config/uplift-bind-plugin.rb
restorecon -v /var/www/stickshift/broker/config/environments/plugin-config/uplift-bind-plugin.rb

perl -p -i -e "s/.*:domain_suffix.*/    :domain_suffix =>  \"${domain}\",/" /var/www/stickshift/broker/config/environments/*.rb

Test

In order to make sure everything is fine after a fresh boot, we recommend to reboot.

After your machine has been rebooted, log in, and try the following

curl -Ik https://localhost/broker/rest/api 
  • If you get an error do
curl -k https://localhost/broker/rest/api