From Fedora Project Wiki

No edit summary
 
(39 intermediate revisions by the same user not shown)
Line 3: Line 3:
so we can edit and adjust  
so we can edit and adjust  


== TODO ==
* complete documentation for node
* add more test
== Setup the Broker ==
The broker is the component of openshift that will dispatch the creation of each application. It consist of a rails application and can be used with various plugin for serving DNS, storing authentication information and communicating with nodes.
=== Install OpenShift Origin packages ===


1 - Install OpenShift Origin packages
<pre>
<pre>
yum install "*openshift*"
yum install openshift-origin-broker openshift-origin-cartridge-diy
yum install mcollective-qpid-plugin
yum install qpid-cpp-server
</pre>
</pre>


2 - Turn off selinux (for now, this step will go away)
Turn off selinux (for now, this step will go away)
<pre>
<pre>
setenforce 0
setenforce 0
Line 17: Line 23:
</pre>
</pre>


3 - Setup the Broker
{{admon/note|The goal is to have it enabled in the end|So once everything is working, please turn it on again and help us to fix the policy}}
3a - Setup MongoDB(This part is the same as the original instructions) https://openshift.redhat.com/community/wiki/build-your-own#Configuring_MongoDB
 
 
=== Setup MongoDB, for authentication and datastore ===
Openshift Origin use mongodb for storing data and as authentication store. So first, we need to install the plugin
<pre>
yum install rubygem-openshift-origin-auth-mongo mongodb-server
</pre>
 
This part is the same as the [https://openshift.redhat.com/community/wiki/build-your-own#Configuring_MongoDB original instructions]
 
Make sure the 2 followings configuration are set in /etc/mongodb.conf :
<pre>
auth = true
smallfiles = true
</pre>
Save and close the file.


    To configure MongoDB to require authentication:
Start mongodb and make sure it starts on reboot
        Open the /etc/mongodb.conf file on the broker host.
        Locate the line beginning with "auth=" and ensure that it is set to "true", as follows:
        auth = true
        Save and close the file.
    To configure the MongoDB default database size:
    Open the /etc/mongodb.conf file on the broker host.
    Locate the line beginning with "smallfiles=" and ensure that it is set to "true", as follows:
    smallfiles = true
    Save and close the file.
    Start mongodb and make sure it starts on reboot
<pre>
<pre>
    systemctl start  mongod.service
systemctl start  mongod
    systemctl enable mongod.service
systemctl enable mongod
</pre>
</pre>
    Make sure the mongo daemon is running by connecting to it
Make sure the mongo daemon is running by connecting to it
    mongo
    Sometimes it takes a little while for it to start up. Keep trying until it connects. Then exit out.
    Create initial mongodb accounts.
<pre>
<pre>
    mongo stickshift_broker_dev --eval 'db.addUser("stickshift", "mooo")'
mongo
    mongo stickshift_broker_dev --eval 'db.auth_user.update({"_id":"admin"}, {"_id":"admin","user":"admin","password":"2a8462d93a13e51387a5e607cbd1139f"}, true)'
</pre>
</pre>


3b - Setup the firewall
Sometimes it takes a little while for it to start up. Keep trying until it connects. Then exit out.
 
Create initial mongodb accounts.
<pre>
mongo stickshift_broker_dev --eval 'db.addUser("stickshift", "mooo")'
mongo stickshift_broker_dev --eval 'db.auth_user.update({"_id":"admin"}, {"_id":"admin","user":"admin","password":"2a8462d93a13e51387a5e607cbd1139f"}, true)'
</pre>
 
=== Setup the firewall ===
<pre>
<pre>
firewall-cmd --add-service=ssh
firewall-cmd --add-service=ssh
firewall-cmd --add-service=https
firewall-cmd --add-service=https
firewall-cmd --add-service=http
firewall-cmd --add-service=http
firewall-cmd --add-service=dns
firewall-cmd --add-port=5672/tcp
</pre>
</pre>


3c - Setup services
=== Setup services ===
<pre>
<pre>
chkconfig network on
systemctl enable httpd
systemctl enable httpd
systemctl enable mcollective
systemctl enable mongod
chkconfig network on
systemctl enable openshift-origin-broker
systemctl enable openshift-origin-broker
systemctl enable sshd
systemctl enable sshd
</pre>
=== Setup mcollective, for broker communication ===
Install the needed rpms for broker communication
<pre>
yum install rubygem-openshift-origin-msg-broker-mcollective mcollective-qpid-plugin qpid-cpp-server
systemctl enable qpidd
systemctl enable qpidd
systemctl start qpidd
systemctl enable mcollective
</pre>
Open the firewall
<pre>
firewall-cmd --add-port=5672/tcp
</pre>
</pre>


3d - Setup mcollective
Install the needed rpm on the node ( same computer is fine )
<pre>
<pre>
yum install openshift-origin-msg-node-mcollective
</pre>
Save off /etc/mcollective/client.cfg and /etc/mcollective/server.cfg. Then edit them so that they look like the following.
Save off /etc/mcollective/client.cfg and /etc/mcollective/server.cfg. Then edit them so that they look like the following.
Note: Change broker.example.com to whatever your hostname is.
Note: Change broker.example.com to whatever your hostname is. Make also sure that daemmonize is set to 1 (see https://bugzilla.redhat.com/show_bug.cgi?id=868417)


/etc/mcollective/client.cfg
/etc/mcollective/client.cfg
 
<pre>
topicprefix = /topic/
topicprefix = /topic/
main_collective = mcollective
main_collective = mcollective
Line 89: Line 116:
factsource = yaml
factsource = yaml
plugin.yaml = /etc/mcollective/facts.yaml
plugin.yaml = /etc/mcollective/facts.yaml
</pre>


/etc/mcollective/server.cfg
/etc/mcollective/server.cfg
<pre>


topicprefix = /topic/
topicprefix = /topic/
Line 112: Line 141:
factsource = yaml
factsource = yaml
plugin.yaml = /etc/mcollective/facts.yaml
plugin.yaml = /etc/mcollective/facts.yaml
</pre>


4e - Setup DNS
=== Setup DNS, based on bind ===
Note1: Do the DNS all in one sweep so the variables match Note2: Change example.com to whatever your machines domain name is. Note3: Yep, except for the first line, you can cut and paste this whole thing.


Install the needed rpms
<pre>
yum install rubygem-openshift-origin-dns-bind
</pre>
Note 1: Do the DNS all in one sweep so the variables match
Note 2: Change example.com to whatever your machines domain name is.
Note 3: Yep, except for the first line, you can cut and paste this whole thing.
<pre>
export domain=example.com
export domain=example.com
export keyfile=/var/named/${domain}.key
export keyfile=/var/named/${domain}.key
Line 130: Line 170:


echo "forwarders { 8.8.8.8; 8.8.4.4; } ;" > /var/named/forwarders.conf
echo "forwarders { 8.8.8.8; 8.8.4.4; } ;" > /var/named/forwarders.conf
restorecon -v /var/named/forwarders.conf
chmod -v 755 /var/named/forwarders.conf
chmod -v 755 /var/named/forwarders.conf


export uplift="$(rpm -q rubygem-openshift-origin-dns-bind --qf '%{NAME}-%{VERSION}')"
sed "s/example.com/${domain}/g" < /usr/share/gems/gems/openshift-origin-dns-bind-*/doc/examples/example.com.db > /var/named/dynamic/${domain}.db
sed "s/example.com/${domain}/g" < /usr/share/gems/gems/${uplift#rubygem-}/doc/examples/example.com.db > /var/named/dynamic/${domain}.db


cat < /var/named/${domain}.key
cat > /var/named/${domain}.key <<EOF
key ${domain} {
key ${domain} {
   algorithm HMAC-MD5;
   algorithm HMAC-MD5;
Line 146: Line 184:


mv /etc/named.conf /etc/named.conf.backup
mv /etc/named.conf /etc/named.conf.backup
sed "s/example.com/${domain}/g" < /usr/share/doc/${uplift}/examples/named.conf > /etc/named.conf
sed "s/example.com/${domain}/g" < /usr/share/doc/rubygem-openshift-origin-dns-bind-*/examples/named.conf > /etc/named.conf
chown -v root:named /etc/named.conf
chown -v root:named /etc/named.conf
/usr/bin/chcon system_u:object_r:named_conf_t:s0 -v /etc/named.conf
restorecon -v /etc/named.conf


/bin/systemctl start named.service
/bin/systemctl start named
 
</pre>
echo "nameserver 127.0.0.1


Then run "nsupdate -k ${keyfile}" and put in the following
Then run "nsupdate -k ${keyfile}" and put in the following
Note1: Change broker.example.com to your hostname
Note1: Change broker.example.com to your hostname
Note2: Change 10.0.0.1 to your ip address Note3: Type CTR+D (The Control key with the "d" key) to exit out of the program.
Note2: Change 10.0.0.1 to your ip address  
 
Note3: Type CTR+D (The Control key with the "d" key) to exit out of the program.
<pre>
server 127.0.0.1
server 127.0.0.1
update delete broker.example.com A
update delete broker.example.com A
update add broker.example.com 180 A 10.0.0.1
update add broker.example.com 180 A 10.0.0.1
send
send
</pre>


Finally, add "nameserver 127.0.0.1" to the top of /etc/resolve.conf
Finally, add "nameserver 127.0.0.1" to the top of /etc/resolv.conf, or add it to Networkmanager configuration.


4f - Setup Plugins
Open the firewall
<pre>
firewall-cmd --add-service=dns
</pre>
 
=== Setup Broker plugins ===
Edit /var/www/stickshift/broker/Gemfile and add the following in the plugin section
Edit /var/www/stickshift/broker/Gemfile and add the following in the plugin section
 
<pre>
gem 'openshift-origin-msg-broker-mcollective'
gem 'openshift-origin-msg-broker-mcollective'
gem 'openshift-origin-dns-bind'
gem 'openshift-origin-dns-bind'
gem 'openshift-origin-auth-mongo'
gem 'openshift-origin-auth-mongo'
 
</pre>
Then do the following
Then do the following
 
<pre>
cd /var/www/stickshift/broker/
cd /var/www/stickshift/broker/
bundle --local
bundle --local
</pre>


Then edit /var/www/stickshift/broker/config/environments/development.rb and add the following at the very last, after the very last 'end' statement
Then edit /var/www/stickshift/broker/config/environments/development.rb and add the following at the very last, after the very last 'end' statement
 
<pre>
require File.expand_path('../plugin-config/openshift-origin-msg-broker-mcollective.rb', __FILE__)
require File.expand_path('../plugin-config/openshift-origin-msg-broker-mcollective.rb', __FILE__)
require File.expand_path('../plugin-config/openshift-origin-dns-bind.rb', __FILE__)
require File.expand_path('../plugin-config/openshift-origin-dns-bind.rb', __FILE__)
require File.expand_path('../plugin-config/openshift-origin-auth-mongo.rb', __FILE__)
require File.expand_path('../plugin-config/openshift-origin-auth-mongo.rb', __FILE__)
 
</pre>
And then do the following.
And then do the following.
It is best if you are still in the same session that you did the DNS setup, so that you have the variables still set.
It is best if you are still in the same session that you did the DNS setup, so that you have the variables still set.
 
<pre>
mkdir -p /var/www/stickshift/broker/config/environments/plugin-config
mkdir -p /var/www/stickshift/broker/config/environments/plugin-config
cat < /var/www/stickshift/broker/config/environments/plugin-config/uplift-bind-plugin.rb  
cat > /var/www/stickshift/broker/config/environments/plugin-config/uplift-bind-plugin.rb <EOF
Broker::Application.configure do
Broker::Application.configure do
   config.dns = {
   config.dns = {
Line 203: Line 248:


perl -p -i -e "s/.*:domain_suffix.*/    :domain_suffix =>  \"${domain}\",/" /var/www/stickshift/broker/config/environments/*.rb
perl -p -i -e "s/.*:domain_suffix.*/    :domain_suffix =>  \"${domain}\",/" /var/www/stickshift/broker/config/environments/*.rb
</pre>
=== Test ===
In order to make sure everything is fine after a fresh boot, we recommend to reboot.


5 - Reboot and Test
After your machine has been rebooted, log in, and try the following
After your machine has been rebooted, log in, and try the following
 
<pre>
curl -Ik https://localhost/broker/rest/api  
curl -Ik https://localhost/broker/rest/api  
</pre>
* If you get an error do
* If you get an error do
<pre>
curl -k https://localhost/broker/rest/api
curl -k https://localhost/broker/rest/api
</pre>

Latest revision as of 13:21, 20 October 2012

Wiki version of https://openshift.redhat.com/community/forums/openshift/fedora-18-openshift-origin-setup-steps-and-testing

so we can edit and adjust

TODO

  • complete documentation for node
  • add more test

Setup the Broker

The broker is the component of openshift that will dispatch the creation of each application. It consist of a rails application and can be used with various plugin for serving DNS, storing authentication information and communicating with nodes.

Install OpenShift Origin packages

yum install openshift-origin-broker openshift-origin-cartridge-diy 

Turn off selinux (for now, this step will go away)

setenforce 0
sed -i -e 's|SELINUX=enforcing|SELINUX=permissive|' /etc/sysconfig/selinux
Note.png
The goal is to have it enabled in the end
So once everything is working, please turn it on again and help us to fix the policy


Setup MongoDB, for authentication and datastore

Openshift Origin use mongodb for storing data and as authentication store. So first, we need to install the plugin

yum install rubygem-openshift-origin-auth-mongo mongodb-server

This part is the same as the original instructions

Make sure the 2 followings configuration are set in /etc/mongodb.conf :

auth = true
smallfiles = true

Save and close the file.

Start mongodb and make sure it starts on reboot

systemctl start  mongod
systemctl enable mongod

Make sure the mongo daemon is running by connecting to it

mongo

Sometimes it takes a little while for it to start up. Keep trying until it connects. Then exit out.

Create initial mongodb accounts.

mongo stickshift_broker_dev --eval 'db.addUser("stickshift", "mooo")'
mongo stickshift_broker_dev --eval 'db.auth_user.update({"_id":"admin"}, {"_id":"admin","user":"admin","password":"2a8462d93a13e51387a5e607cbd1139f"}, true)'

Setup the firewall

firewall-cmd --add-service=ssh
firewall-cmd --add-service=https
firewall-cmd --add-service=http

Setup services

chkconfig network on
systemctl enable httpd
systemctl enable openshift-origin-broker
systemctl enable sshd

Setup mcollective, for broker communication

Install the needed rpms for broker communication

yum install rubygem-openshift-origin-msg-broker-mcollective mcollective-qpid-plugin qpid-cpp-server
systemctl enable qpidd
systemctl start qpidd
systemctl enable mcollective

Open the firewall

firewall-cmd --add-port=5672/tcp

Install the needed rpm on the node ( same computer is fine )

yum install openshift-origin-msg-node-mcollective 

Save off /etc/mcollective/client.cfg and /etc/mcollective/server.cfg. Then edit them so that they look like the following. Note: Change broker.example.com to whatever your hostname is. Make also sure that daemmonize is set to 1 (see https://bugzilla.redhat.com/show_bug.cgi?id=868417)

/etc/mcollective/client.cfg

topicprefix = /topic/
main_collective = mcollective
collectives = mcollective
libdir = /usr/libexec/mcollective
loglevel = debug
logfile = /var/log/mcollective-client.log

# Plugins
securityprovider = psk
plugin.psk = unset
connector = qpid
plugin.qpid.host=broker.example.com
plugin.qpid.secure=false
plugin.qpid.timeout=5

# Facts
factsource = yaml
plugin.yaml = /etc/mcollective/facts.yaml

/etc/mcollective/server.cfg


topicprefix = /topic/
main_collective = mcollective
collectives = mcollective
libdir = /usr/libexec/mcollective
logfile = /var/log/mcollective.log
loglevel = debug
daemonize = 1
direct_addressing = n

# Plugins
securityprovider = psk
plugin.psk = unset
connector = qpid
plugin.qpid.host=broker.example.com
plugin.qpid.secure=false
plugin.qpid.timeout=5

# Facts
factsource = yaml
plugin.yaml = /etc/mcollective/facts.yaml

Setup DNS, based on bind

Install the needed rpms

yum install rubygem-openshift-origin-dns-bind

Note 1: Do the DNS all in one sweep so the variables match

Note 2: Change example.com to whatever your machines domain name is.

Note 3: Yep, except for the first line, you can cut and paste this whole thing.

export domain=example.com
export keyfile=/var/named/${domain}.key

rm -vf /var/named/K${domain}*
cd /var/named
dnssec-keygen -a HMAC-MD5 -b 512 -n USER -r /dev/urandom ${domain}
KEY="$(grep Key: K${domain}*.private | cut -d ' ' -f 2)"

rndc-confgen -a -r /dev/urandom
restorecon -v /etc/rndc.* /etc/named.*
chown -v root:named /etc/rndc.key
chmod -v 640 /etc/rndc.key

echo "forwarders { 8.8.8.8; 8.8.4.4; } ;" > /var/named/forwarders.conf
chmod -v 755 /var/named/forwarders.conf

sed "s/example.com/${domain}/g" < /usr/share/gems/gems/openshift-origin-dns-bind-*/doc/examples/example.com.db > /var/named/dynamic/${domain}.db

cat > /var/named/${domain}.key <<EOF
key ${domain} {
  algorithm HMAC-MD5;
  secret "${KEY}";
};
EOF
chown -Rv named:named /var/named
restorecon -rv /var/named

mv /etc/named.conf /etc/named.conf.backup
sed "s/example.com/${domain}/g" < /usr/share/doc/rubygem-openshift-origin-dns-bind-*/examples/named.conf > /etc/named.conf
chown -v root:named /etc/named.conf
restorecon -v /etc/named.conf

/bin/systemctl start named

Then run "nsupdate -k ${keyfile}" and put in the following Note1: Change broker.example.com to your hostname Note2: Change 10.0.0.1 to your ip address Note3: Type CTR+D (The Control key with the "d" key) to exit out of the program.

server 127.0.0.1
update delete broker.example.com A
update add broker.example.com 180 A 10.0.0.1
send

Finally, add "nameserver 127.0.0.1" to the top of /etc/resolv.conf, or add it to Networkmanager configuration.

Open the firewall

firewall-cmd --add-service=dns

Setup Broker plugins

Edit /var/www/stickshift/broker/Gemfile and add the following in the plugin section

gem 'openshift-origin-msg-broker-mcollective'
gem 'openshift-origin-dns-bind'
gem 'openshift-origin-auth-mongo'

Then do the following

cd /var/www/stickshift/broker/
bundle --local

Then edit /var/www/stickshift/broker/config/environments/development.rb and add the following at the very last, after the very last 'end' statement

require File.expand_path('../plugin-config/openshift-origin-msg-broker-mcollective.rb', __FILE__)
require File.expand_path('../plugin-config/openshift-origin-dns-bind.rb', __FILE__)
require File.expand_path('../plugin-config/openshift-origin-auth-mongo.rb', __FILE__)

And then do the following. It is best if you are still in the same session that you did the DNS setup, so that you have the variables still set.

mkdir -p /var/www/stickshift/broker/config/environments/plugin-config
cat > /var/www/stickshift/broker/config/environments/plugin-config/uplift-bind-plugin.rb <EOF
Broker::Application.configure do
  config.dns = {
  :server => "127.0.0.1",
  :port => 53,
  :keyname => "${domain}",
  :keyvalue => "${KEY}",
  :zone => "${domain}"
}
end
EOF

chown -v apache:apache /var/www/stickshift/broker/config/environments/plugin-config/uplift-bind-plugin.rb
restorecon -v /var/www/stickshift/broker/config/environments/plugin-config/uplift-bind-plugin.rb

perl -p -i -e "s/.*:domain_suffix.*/    :domain_suffix =>  \"${domain}\",/" /var/www/stickshift/broker/config/environments/*.rb

Test

In order to make sure everything is fine after a fresh boot, we recommend to reboot.

After your machine has been rebooted, log in, and try the following

curl -Ik https://localhost/broker/rest/api 
  • If you get an error do
curl -k https://localhost/broker/rest/api