OpenShift Origin-F19

From FedoraProject

(Difference between revisions)
Jump to: navigation, search
 
(33 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
Fedora 19 is when OpenShift Origin first became a feature.
 
Fedora 19 is when OpenShift Origin first became a feature.
 +
 +
NOTE: (August 8, 2013) This page is getting an update.  It will accommodate F19 cloud images (not just minimal install).  It is also updated with the OpenShift Origin Version 2 documentation.
  
 
This page is here to show how to setup OpenShift Origin on Fedora 19 using the packages in Fedora, as opposed to the packages published from upstream.  These steps are written out to be done by hand.  Yes, people can script and/or puppetize these steps.  But these are written out so that people can see, and fine tune them.
 
This page is here to show how to setup OpenShift Origin on Fedora 19 using the packages in Fedora, as opposed to the packages published from upstream.  These steps are written out to be done by hand.  Yes, people can script and/or puppetize these steps.  But these are written out so that people can see, and fine tune them.
 +
 +
Note: And now they have been written into scripts. https://github.com/tdawson/oo-install-scripts
  
 
Goal: By the end of this, you should have two machines.  A broker machine, and one node machine.  You should be able to create applications, that will be put on the node machine.  You should be able to check the status of those applications.  You should be able to point your web browser to the URL of those applications.
 
Goal: By the end of this, you should have two machines.  A broker machine, and one node machine.  You should be able to create applications, that will be put on the node machine.  You should be able to check the status of those applications.  You should be able to point your web browser to the URL of those applications.
Line 11: Line 15:
 
* https://www.openshift.com/forums/openshift/fedora-18-openshift-origin-setup-steps-and-testing
 
* https://www.openshift.com/forums/openshift/fedora-18-openshift-origin-setup-steps-and-testing
  
= Initial Setup of Broker and Node Machines =
+
= '''''Initial Setup of Broker and Node Machines''''' =
  
 
'''ON BOTH BROKER AND NODE'''
 
'''ON BOTH BROKER AND NODE'''
Line 40: Line 44:
 
  nm-tool | grep Address | grep -v HW | awk '{print $2}'
 
  nm-tool | grep Address | grep -v HW | awk '{print $2}'
  
= Setup and Configure Broker =
+
= '''''Setup and Configure Broker''''' =
  
== Broker: Bind DNS ==
+
== '''Broker: Bind DNS''' ==
  
 
  yum -y install bind bind-utils
 
  yum -y install bind bind-utils
Line 48: Line 52:
 
  KEYFILE=/var/named/${DOMAIN}.key
 
  KEYFILE=/var/named/${DOMAIN}.key
  
 +
setup DNSSEC key pair
 
  cd /var/named/
 
  cd /var/named/
 
  dnssec-keygen -a HMAC-MD5 -b 512 -n USER -r /dev/urandom ${DOMAIN}
 
  dnssec-keygen -a HMAC-MD5 -b 512 -n USER -r /dev/urandom ${DOMAIN}
Line 55: Line 60:
 
  echo $KEY
 
  echo $KEY
  
 +
setup permissions for the DNSSEC key pair
 
  restorecon -v /etc/rndc.* /etc/named.*
 
  restorecon -v /etc/rndc.* /etc/named.*
 
  chown -v root:named /etc/rndc.key
 
  chown -v root:named /etc/rndc.key
 
  chmod -v 640 /etc/rndc.key
 
  chmod -v 640 /etc/rndc.key
  
 +
setup forwarders
 
  echo "forwarders { 8.8.8.8; 8.8.4.4; } ;" >> /var/named/forwarders.conf
 
  echo "forwarders { 8.8.8.8; 8.8.4.4; } ;" >> /var/named/forwarders.conf
 
  restorecon -v /var/named/forwarders.conf
 
  restorecon -v /var/named/forwarders.conf
 
  chmod -v 755 /var/named/forwarders.conf
 
  chmod -v 755 /var/named/forwarders.conf
  
 +
setup initial DNS database
 
  rm -rvf /var/named/dynamic
 
  rm -rvf /var/named/dynamic
 
  mkdir -vp /var/named/dynamic
 
  mkdir -vp /var/named/dynamic
 
+
 
  cat <<EOF > /var/named/dynamic/${DOMAIN}.db
 
  cat <<EOF > /var/named/dynamic/${DOMAIN}.db
 
  \$ORIGIN .
 
  \$ORIGIN .
Line 83: Line 91:
 
  EOF
 
  EOF
  
  cat <<EOF > /var/named/${DOMAIN}.key
+
Install the DNSSEC key
 +
  cat <<EOF > ${KEYFILE}
 
  key ${DOMAIN} {
 
  key ${DOMAIN} {
 
   algorithm HMAC-MD5;
 
   algorithm HMAC-MD5;
Line 90: Line 99:
 
  EOF
 
  EOF
  
 +
Check the key and database
 
  cat /var/named/dynamic/${DOMAIN}.db
 
  cat /var/named/dynamic/${DOMAIN}.db
 
  cat /var/named/${DOMAIN}.key
 
  cat /var/named/${DOMAIN}.key
+
 
 +
Set permissions for key and database
 
  chown -Rv named:named /var/named
 
  chown -Rv named:named /var/named
 
  restorecon -rv /var/named
 
  restorecon -rv /var/named
  
 +
Create the named configuration file
 
  mv /etc/named.conf /etc/named.conf.openshift
 
  mv /etc/named.conf /etc/named.conf.openshift
 
  cat <<EOF > /etc/named.conf
 
  cat <<EOF > /etc/named.conf
Line 118: Line 130:
 
     bindkeys-file "/etc/named.iscdlv.key";
 
     bindkeys-file "/etc/named.iscdlv.key";
 
   
 
   
     // set forwarding to the next nearest server (from DHCP response
+
     // set forwarding to the next nearest server (from DHCP response)
 
     forward only;
 
     forward only;
 
     include "forwarders.conf";
 
     include "forwarders.conf";
Line 148: Line 160:
 
  };
 
  };
 
  EOF
 
  EOF
+
 
 +
Check the named config file
 
  cat /etc/named.conf
 
  cat /etc/named.conf
 +
 +
setup permissions of named config file
 
  chown -v root:named /etc/named.conf
 
  chown -v root:named /etc/named.conf
 
  restorecon /etc/named.conf
 
  restorecon /etc/named.conf
+
 
 +
Setup firewall
 
  firewall-cmd --add-service=dns
 
  firewall-cmd --add-service=dns
 
  firewall-cmd --permanent --add-service=dns
 
  firewall-cmd --permanent --add-service=dns
 
  firewall-cmd --list-all
 
  firewall-cmd --list-all
 +
 +
Setup and start service
 
  /bin/systemctl enable named.service
 
  /bin/systemctl enable named.service
 
  /bin/systemctl start named.service
 
  /bin/systemctl start named.service
  
 +
add entries using nsupdate
 
  nsupdate -k ${KEYFILE}
 
  nsupdate -k ${KEYFILE}
 
  > server 127.0.0.1
 
  > server 127.0.0.1
Line 167: Line 186:
 
  > quit
 
  > quit
  
 +
Test DNS server
 +
This is best done before hostname has been set.
 
  ping broker.example.com
 
  ping broker.example.com
 
  dig @127.0.0.1 broker.example.com
 
  dig @127.0.0.1 broker.example.com
 
  
== Broker: DHCP client and hostname ==
+
== '''Broker: DHCP client and hostname''' ==
  
 +
Setup dhcp client
 
  echo "prepend domain-name-servers **your broker ip address**;" >> /etc/dhcp/dhclient-eth0.conf
 
  echo "prepend domain-name-servers **your broker ip address**;" >> /etc/dhcp/dhclient-eth0.conf
 
  echo "supersede host-name \"broker\";" >> /etc/dhcp/dhclient-eth0.conf
 
  echo "supersede host-name \"broker\";" >> /etc/dhcp/dhclient-eth0.conf
 
  echo "supersede domain-name \"example.com\";" >> /etc/dhcp/dhclient-eth0.conf
 
  echo "supersede domain-name \"example.com\";" >> /etc/dhcp/dhclient-eth0.conf
  
== Broker: hostname ==
+
Setup hostname
 
+
 
  echo "broker.example.com" > /etc/hostname
 
  echo "broker.example.com" > /etc/hostname
  
== Broker: MongoDB ==
+
== '''Broker: MongoDB''' ==
 
+
Install Software
 
  yum -y install mongodb-server
 
  yum -y install mongodb-server
  
 +
Tweak config file
 
  vi /etc/mongodb.conf
 
  vi /etc/mongodb.conf
 
  # Uncomment auth = true
 
  # Uncomment auth = true
 
  # Add smallfiles = true
 
  # Add smallfiles = true
  
 +
Setup and start service
 
  /usr/bin/systemctl enable mongod.service
 
  /usr/bin/systemctl enable mongod.service
 
  /usr/bin/systemctl status mongod.service
 
  /usr/bin/systemctl status mongod.service
Line 194: Line 216:
 
  /usr/bin/systemctl status mongod.service
 
  /usr/bin/systemctl status mongod.service
  
# Testing
+
Testing
 
  mongo
 
  mongo
 
  > show dbs
 
  > show dbs
 
  > exit
 
  > exit
  
== Broker: QPID ==
+
== '''Broker: Messaging (using QPID)''' ==
 
Activemq on F19 isn't ready for OpenShift production.  When it is, we'll use that
 
Activemq on F19 isn't ready for OpenShift production.  When it is, we'll use that
 
For now we'll use QPID with mcollective.
 
For now we'll use QPID with mcollective.
  
 +
Install Software
 
  yum install mcollective-qpid-plugin qpid-cpp-server
 
  yum install mcollective-qpid-plugin qpid-cpp-server
 +
 +
Setup Firewall
 
  firewall-cmd --add-port=5672/tcp
 
  firewall-cmd --add-port=5672/tcp
 
  firewall-cmd --permanent --add-port=5672/tcp
 
  firewall-cmd --permanent --add-port=5672/tcp
 
  firewall-cmd --list-all
 
  firewall-cmd --list-all
  
 +
Setup and start service
 
  /usr/bin/systemctl enable qpidd.service
 
  /usr/bin/systemctl enable qpidd.service
 
  /usr/bin/systemctl start qpidd.service
 
  /usr/bin/systemctl start qpidd.service
 
  /usr/bin/systemctl status qpidd.service
 
  /usr/bin/systemctl status qpidd.service
  
== Broker: MCollective client ( using QPID) ==
+
== '''Broker: MCollective client ( using QPID)''' ==
 +
Install Software
 
  yum -y install mcollective-client
 
  yum -y install mcollective-client
  
 +
Move original config file out of the way
 
  mv /etc/mcollective/client.cfg /etc/mcollective/client.cfg.orig
 
  mv /etc/mcollective/client.cfg /etc/mcollective/client.cfg.orig
   
+
 
 +
Create new client config file. This config file is for using QPID as a messaging platform.
 
  cat <<EOF > /etc/mcollective/client.cfg
 
  cat <<EOF > /etc/mcollective/client.cfg
 
  topicprefix = /topic/
 
  topicprefix = /topic/
Line 229: Line 258:
 
  plugin.psk = unset
 
  plugin.psk = unset
 
  connector = qpid
 
  connector = qpid
  plugin.qpid.host=broker.example.com
+
  plugin.qpid.host=${BROKERNAME}
 
  plugin.qpid.secure=false
 
  plugin.qpid.secure=false
 
  plugin.qpid.timeout=5
 
  plugin.qpid.timeout=5
Line 238: Line 267:
 
  EOF
 
  EOF
  
== Broker: broker application ==
+
== '''Broker: broker application''' ==
 +
Install software
 
  yum -y install openshift-origin-broker openshift-origin-broker-util rubygem-openshift-origin-auth-remote-user rubygem-openshift-origin-msg-broker-mcollective rubygem-openshift-origin-dns-bind
 
  yum -y install openshift-origin-broker openshift-origin-broker-util rubygem-openshift-origin-auth-remote-user rubygem-openshift-origin-msg-broker-mcollective rubygem-openshift-origin-dns-bind
  
 +
Modify the broker proxy server name
 
  sed -i -e "s/ServerName .*$/ServerName broker.example.com/" /etc/httpd/conf.d/000002_openshift_origin_broker_servername.conf  
 
  sed -i -e "s/ServerName .*$/ServerName broker.example.com/" /etc/httpd/conf.d/000002_openshift_origin_broker_servername.conf  
 
  cat /etc/httpd/conf.d/000002_openshift_origin_broker_servername.conf
 
  cat /etc/httpd/conf.d/000002_openshift_origin_broker_servername.conf
  
 +
Setup and start service
 
  /usr/bin/systemctl enable httpd.service
 
  /usr/bin/systemctl enable httpd.service
 
  /usr/bin/systemctl enable ntpd.service
 
  /usr/bin/systemctl enable ntpd.service
 
  /usr/bin/systemctl enable sshd.service
 
  /usr/bin/systemctl enable sshd.service
  
 +
Setup Firewall
 
  firewall-cmd --add-service=ssh
 
  firewall-cmd --add-service=ssh
 
  firewall-cmd --add-service=http
 
  firewall-cmd --add-service=http
Line 256: Line 289:
 
  firewall-cmd --list-all
 
  firewall-cmd --list-all
  
 +
Generate access key
 
  openssl genrsa -out /etc/openshift/server_priv.pem 2048
 
  openssl genrsa -out /etc/openshift/server_priv.pem 2048
 
  openssl rsa -in /etc/openshift/server_priv.pem -pubout > /etc/openshift/server_pub.pem
 
  openssl rsa -in /etc/openshift/server_priv.pem -pubout > /etc/openshift/server_pub.pem
Line 261: Line 295:
 
  cp -v ~/.ssh/rsync_id_rsa* /etc/openshift/
 
  cp -v ~/.ssh/rsync_id_rsa* /etc/openshift/
  
 +
Setup selinux boolean variables and set file contexts
 
  setsebool -P httpd_unified=on httpd_can_network_connect=on httpd_can_network_relay=on httpd_run_stickshift=on named_write_master_zones=on
 
  setsebool -P httpd_unified=on httpd_can_network_connect=on httpd_can_network_relay=on httpd_run_stickshift=on named_write_master_zones=on
 
  fixfiles -R rubygem-passenger restore
 
  fixfiles -R rubygem-passenger restore
Line 267: Line 302:
 
  restorecon -rv /usr/share/gems/gems/passenger-*
 
  restorecon -rv /usr/share/gems/gems/passenger-*
  
 +
Tweak broker config, if needed
 
  vi /etc/openshift/broker.conf
 
  vi /etc/openshift/broker.conf
 
  # Might not have to do anything but make sure you have the following lines
 
  # Might not have to do anything but make sure you have the following lines
Line 272: Line 308:
 
  VALID_GEAR_SIZES="small,medium"
 
  VALID_GEAR_SIZES="small,medium"
  
== Broker: broker plugins and MongoDB user accounts ==
+
== '''Broker: broker plugins and MongoDB user accounts''' ==
 
+
Create config files from examples
cp /usr/share/gems/gems/openshift-origin-auth-remote-user-*/conf/openshift-origin- auth-remote-user.conf.example /etc/openshift/plugins.d/openshift-origin-auth-remote-user.conf
+
 
  cp /etc/openshift/plugins.d/openshift-origin-msg-broker-mcollective.conf.example /etc/openshift/plugins.d/openshift-origin-msg-broker-mcollective.conf
 
  cp /etc/openshift/plugins.d/openshift-origin-msg-broker-mcollective.conf.example /etc/openshift/plugins.d/openshift-origin-msg-broker-mcollective.conf
  
 +
Config the DNS plugin
 
  cd /var/named/
 
  cd /var/named/
 
  KEY="$(grep Key: K${DOMAIN}*.private | cut -d ' ' -f 2)"
 
  KEY="$(grep Key: K${DOMAIN}*.private | cut -d ' ' -f 2)"
 
  cat $KEYFILE
 
  cat $KEYFILE
 
  echo $KEY
 
  echo $KEY
 
+
 
  cat <<EOF > /etc/openshift/plugins.d/openshift-origin-dns-bind.conf
 
  cat <<EOF > /etc/openshift/plugins.d/openshift-origin-dns-bind.conf
 
  BIND_SERVER="127.0.0.1"
 
  BIND_SERVER="127.0.0.1"
Line 290: Line 326:
 
  EOF
 
  EOF
  
 +
Configure authentication plugin and add a user
 
  cp -v /var/www/openshift/broker/httpd/conf.d/openshift-origin-auth-remote-user-basic.conf.sample /var/www/openshift/broker/httpd/conf.d/openshift-origin-auth-remote-user.conf
 
  cp -v /var/www/openshift/broker/httpd/conf.d/openshift-origin-auth-remote-user-basic.conf.sample /var/www/openshift/broker/httpd/conf.d/openshift-origin-auth-remote-user.conf
 
  htpasswd -c -b -s /etc/openshift/htpasswd demo demo
 
  htpasswd -c -b -s /etc/openshift/htpasswd demo demo
Line 295: Line 332:
 
  cat /etc/openshift/htpasswd
 
  cat /etc/openshift/htpasswd
  
 +
Add MongoDB account
 
  grep MONGO /etc/openshift/broker.conf
 
  grep MONGO /etc/openshift/broker.conf
 
  mongo openshift_broker_dev --eval 'db.addUser("openshift", "mooo")'
 
  mongo openshift_broker_dev --eval 'db.addUser("openshift", "mooo")'
 
  # If you are going to change the username and/or password, change broker.conf
 
  # If you are going to change the username and/or password, change broker.conf
  
  yum -y install rubygem-psych
+
Bundle broker gems
 +
  yum -y install rubygem-psych rubygem-mocha
 
  cd /var/www/openshift/broker
 
  cd /var/www/openshift/broker
 
  gem install mongoid
 
  gem install mongoid
 
  bundle --local
 
  bundle --local
  
 +
Setup and start services
 
  /usr/bin/systemctl enable openshift-broker.service
 
  /usr/bin/systemctl enable openshift-broker.service
 
  /usr/bin/systemctl start httpd.service
 
  /usr/bin/systemctl start httpd.service
Line 309: Line 349:
 
  /usr/bin/systemctl status openshift-broker.service
 
  /usr/bin/systemctl status openshift-broker.service
  
  curl -k -u demo:demopassword https://localhost/broker/rest/api
+
Test basic broker service
 +
  curl -k -u demo:demo https://localhost/broker/rest/api
  
= Setup and Configure Node =
+
= '''''Setup and Configure Node''''' =
  
== Node: Initial setup/configure ==
+
== '''Node: Initial setup/configure''' ==
 
'''ON BROKER'''
 
'''ON BROKER'''
  keyfile=/var/named/${DOMAIN}.key
+
  KEYFILE=/var/named/${DOMAIN}.key
  oo-register-dns -h node -d ${DOMAIN} -n ${NODEIP} -k ${keyfile}
+
 
 +
Register the node in DNS
 +
  oo-register-dns -h ${NODENAME} -d ${DOMAIN} -n ${NODEIP} -k ${KEYFILE}
  
 +
Copy the broker public key to node
 
  scp /etc/openshift/rsync_id_rsa.pub root@${NODENAME}:/root/.ssh/
 
  scp /etc/openshift/rsync_id_rsa.pub root@${NODENAME}:/root/.ssh/
  
 
'''ON NODE'''
 
'''ON NODE'''
 +
Put the brokers public key in root authorized keys
 
  cat /root/.ssh/rsync_id_rsa.pub >> /root/.ssh/authorized_keys
 
  cat /root/.ssh/rsync_id_rsa.pub >> /root/.ssh/authorized_keys
 
  rm -f /root/.ssh/rsync_id_rsa.pub
 
  rm -f /root/.ssh/rsync_id_rsa.pub
  
 
'''ON BROKER'''
 
'''ON BROKER'''
 +
Test to make sure we can login using our key
 
  ssh -i /root/.ssh/rsync_id_rsa root@${NODENAME}
 
  ssh -i /root/.ssh/rsync_id_rsa root@${NODENAME}
 
  exit
 
  exit
  
== Node: DHCP client ==
+
== '''Node: DHCP client and hostname''' ==
 +
Configure the dhcp settings
 
  echo "prepend domain-name-servers **your broker ip address**;" >> /etc/dhcp/dhclient-eth0.conf
 
  echo "prepend domain-name-servers **your broker ip address**;" >> /etc/dhcp/dhclient-eth0.conf
 
  echo "supersede host-name \"node\";" >> /etc/dhcp/dhclient-eth0.conf
 
  echo "supersede host-name \"node\";" >> /etc/dhcp/dhclient-eth0.conf
 
  echo "supersede domain-name \"example.com\";" >> /etc/dhcp/dhclient-eth0.conf
 
  echo "supersede domain-name \"example.com\";" >> /etc/dhcp/dhclient-eth0.conf
  
== Node: hostname ==
+
Set the hostname
 
  echo "node.example.com" > /etc/hostname
 
  echo "node.example.com" > /etc/hostname
  
== Node: MCollective  ==
+
== '''Node: MCollective''' ==
 
'''ON NODE'''
 
'''ON NODE'''
  yum -y install openshift-origin-msg-node-mcollective
+
Install Software
 +
  yum -y install openshift-origin-msg-node-mcollective mcollective-qpid-plugin
  
 +
Move original configuration out of the way
 
  mv /etc/mcollective/server.cfg /etc/mcollective/server.cfg.orig
 
  mv /etc/mcollective/server.cfg /etc/mcollective/server.cfg.orig
+
 
 +
Create new configuration
 
  cat <<EOF > /etc/mcollective/server.cfg
 
  cat <<EOF > /etc/mcollective/server.cfg
 
  topicprefix = /topic/
 
  topicprefix = /topic/
Line 365: Line 415:
 
  EOF
 
  EOF
  
 +
Setup and start services
 
  /bin/systemctl enable mcollective.service
 
  /bin/systemctl enable mcollective.service
 
  /bin/systemctl start  mcollective.service
 
  /bin/systemctl start  mcollective.service
Line 370: Line 421:
 
'''ON BROKER'''
 
'''ON BROKER'''
 
  mco ping
 
  mco ping
 +
# node should show up on mco ping
  
== Node: node application ==
+
== '''Node: node application''' ==
 +
Install software
 
  yum -y install rubygem-openshift-origin-node rubygem-passenger-native openshift-origin-port-proxy openshift-origin-node-util
 
  yum -y install rubygem-openshift-origin-node rubygem-passenger-native openshift-origin-port-proxy openshift-origin-node-util
 
  yum -y install openshift-origin-cartridge-cron-1.4 openshift-origin-cartridge-diy-0.1
 
  yum -y install openshift-origin-cartridge-cron-1.4 openshift-origin-cartridge-diy-0.1
  
 +
Setup firewall
 
  firewall-cmd --add-service=ssh
 
  firewall-cmd --add-service=ssh
 
  firewall-cmd --add-service=http
 
  firewall-cmd --add-service=http
Line 383: Line 437:
 
  firewall-cmd --list-all
 
  firewall-cmd --list-all
  
== Node: PAM namespace module, cgropus, and user quotas ==
+
== '''Node: PAM namespace module, cgroups, and user quotas''' ==
'''ON NODE'''
+
PAM
# PAM
+
 
  sed -i -e 's|pam_selinux|pam_openshift|g' /etc/pam.d/sshd
 
  sed -i -e 's|pam_selinux|pam_openshift|g' /etc/pam.d/sshd
 
   
 
   
Line 397: Line 450:
 
  done
 
  done
  
#CGROUPS
+
CGROUPS
Need to still fixup the cgroup instructions
+
 
  #echo "mount {" >> /etc/cgconfig.conf
+
Cgroups Config - Need to still fixup the cgroup configurations
  #echo "        cpu    = /cgroup/all;" >> /etc/cgconfig.conf
+
  echo "mount {" >> /etc/cgconfig.conf
  #echo "        cpuacct = /cgroup/all;" >> /etc/cgconfig.conf
+
  echo "        cpu    = /cgroup/all;" >> /etc/cgconfig.conf
  #echo "        memory  = /cgroup/all;" >> /etc/cgconfig.conf
+
  echo "        cpuacct = /cgroup/all;" >> /etc/cgconfig.conf
  #echo "        freezer = /cgroup/all;" >> /etc/cgconfig.conf
+
  echo "        memory  = /cgroup/all;" >> /etc/cgconfig.conf
  #echo "        net_cls = /cgroup/all;" >> /etc/cgconfig.conf
+
  echo "        freezer = /cgroup/all;" >> /etc/cgconfig.conf
  #echo "}" >> /etc/cgconfig.conf
+
  echo "        net_cls = /cgroup/all;" >> /etc/cgconfig.conf
  #restorecon -v /etc/cgconfig.conf
+
  echo "}" >> /etc/cgconfig.conf
  #mkdir /cgroup
+
  restorecon -v /etc/cgconfig.conf
  #restorecon -RFvv /cgroup
+
  mkdir /cgroup
 +
  restorecon -RFvv /cgroup
  
 +
Cgroups enable and startup services
 
  /bin/systemctl enable cgconfig.service
 
  /bin/systemctl enable cgconfig.service
 
  /bin/systemctl enable cgred.service
 
  /bin/systemctl enable cgred.service
Line 417: Line 472:
 
  /usr/sbin/service openshift-cgroups restart
 
  /usr/sbin/service openshift-cgroups restart
  
#DISK QUOTA
+
DISK QUOTA
 
  # Edit fstab and add usrquota to whichever filesystem  
 
  # Edit fstab and add usrquota to whichever filesystem  
 
  #  has /var/lib/openshift on it
 
  #  has /var/lib/openshift on it
Line 425: Line 480:
 
  quotacheck -cmug /
 
  quotacheck -cmug /
  
== Node: SELinux and System Control==
+
== '''Node: SELinux and System Control''' ==
'''ON NODE'''
+
Setup SELINUX Booleans
# SELINUX
+
 
  setsebool -P httpd_unified=on httpd_can_network_connect=on httpd_can_network_relay=on httpd_read_user_content=on httpd_enable_homedirs=on httpd_run_stickshift=on allow_polyinstantiation=on
 
  setsebool -P httpd_unified=on httpd_can_network_connect=on httpd_can_network_relay=on httpd_read_user_content=on httpd_enable_homedirs=on httpd_run_stickshift=on allow_polyinstantiation=on
  
 +
Update selinux file setting
 
  restorecon -rv /var/run
 
  restorecon -rv /var/run
 
  restorecon -rv /usr/sbin/mcollectived /var/log/mcollective.log /var/run/mcollectived.pid
 
  restorecon -rv /usr/sbin/mcollectived /var/log/mcollective.log /var/run/mcollectived.pid
 
  restorecon -rv /var/lib/openshift /etc/openshift/node.conf /etc/httpd/conf.d/openshift
 
  restorecon -rv /var/lib/openshift /etc/openshift/node.conf /etc/httpd/conf.d/openshift
  
# SYSTEM CONTROL SETTINGS
+
SYSTEM CONTROL SETTINGS
 
  echo "# Added for OpenShift" >> /etc/sysctl.d/openshift.conf
 
  echo "# Added for OpenShift" >> /etc/sysctl.d/openshift.conf
 
  echo "kernel.sem = 250  32000 32  4096" >> /etc/sysctl.d/openshift.conf
 
  echo "kernel.sem = 250  32000 32  4096" >> /etc/sysctl.d/openshift.conf
Line 441: Line 496:
 
  sysctl -p /etc/sysctl.d/openshift.conf
 
  sysctl -p /etc/sysctl.d/openshift.conf
  
== Node: SSH, Port Proxy, and Node application ==
+
== '''Node: SSH, Port Proxy, and Node application''' ==
'''ON NODE'''
+
SSH
# SSH
+
 
  vi /etc/ssh/sshd_config
 
  vi /etc/ssh/sshd_config
 
  > AcceptEnv GIT_SSH
 
  > AcceptEnv GIT_SSH
Line 452: Line 506:
 
  /bin/systemctl restart  sshd.service
 
  /bin/systemctl restart  sshd.service
  
# PORT PROXY
+
PORT PROXY
 
+
 
  firewall-cmd --add-port=35531-65535/tcp
 
  firewall-cmd --add-port=35531-65535/tcp
 
  firewall-cmd --permanent --add-port=35531-65535/tcp
 
  firewall-cmd --permanent --add-port=35531-65535/tcp
Line 460: Line 513:
 
  /bin/systemctl enable openshift-port-proxy.service
 
  /bin/systemctl enable openshift-port-proxy.service
 
  /bin/systemctl restart  openshift-port-proxy.service
 
  /bin/systemctl restart  openshift-port-proxy.service
+
 
# NODE SETUP
+
NODE SETUP
 +
/bin/systemctl enable httpd.service
 
  /bin/systemctl enable openshift-gears.service
 
  /bin/systemctl enable openshift-gears.service
 
   
 
   
Line 472: Line 526:
 
  /etc/cron.minutely/openshift-facts
 
  /etc/cron.minutely/openshift-facts
  
== Node: Reboot ==
+
== '''Node: Reboot''' ==
 +
We need to reboot to load all the node stuff correctly
 
  reboot
 
  reboot
  
= Test =
+
= '''''Testing''''' =
'''ON BROKER''' (after node is back up)
+
==Test on Broker (after node is back up)==
 +
 
 +
'''Check Messaging'''
 
  mco ping
 
  mco ping
 +
Should look like
 +
node.example.com                        time=239.51 ms
 +
 +
---- ping statistics ----
 +
1 replies max: 239.51 min: 239.51 avg: 239.51
 +
 +
'''Check Broker'''
 
  curl -k -u demo:demo https://localhost/broker/rest/api
 
  curl -k -u demo:demo https://localhost/broker/rest/api
 +
Should look like
 +
{"data":{"API":{"href":"https://localhost/broker/rest/api","method":"GET","optional_params":[],"rel":"API entry point","required_params":[]},"GET_ENVIRONMENT":{"href":"https://localhost/broker/rest/environment","method":"GET","optional_params":[],"rel":"Get environment information","required_params":[]},"GET_USER"
 +
...
 +
:id","type":"string","valid_options":[]}]}},"messages":[],"status":"ok","supported_api_versions":[1.0,1.1,1.2,1.3],"type":"links","version":"1.3"}
  
 +
'''Check and Setup User'''
 
  yum -y install rubygem-rhc
 
  yum -y install rubygem-rhc
 
  LIBRA_SERVER=broker.example.com rhc setup
 
  LIBRA_SERVER=broker.example.com rhc setup
 +
Should look like (Note: Generate a token now? no - client can handle it, broker in F19 cannot)
 +
OpenShift Client Tools (RHC) Setup Wizard
 +
 +
This wizard will help you upload your SSH keys, set your application namespace, and
 +
check that other programs like Git are properly installed.
 +
 +
The server's certificate is self-signed, which means that a secure connection can't be
 +
established to 'broker.example.com'.
 +
 +
You may bypass this check, but any data you send to the server could be intercepted by
 +
others.
 +
 +
Connect without checking the certificate? (yes|no): yes
 +
Login to broker.example.com: demo
 +
Password: ****
 +
 
 +
OpenShift can create and store a token on disk which allows to you to access the
 +
server without using your password. The key is stored in your home directory and
 +
should be kept secret.  You can delete the key at any time by running 'rhc logout'.
 +
Generate a token now? (yes|no) no
 +
 +
Saving configuration to /root/.openshift/express.conf ... done
 +
 +
No SSH keys were found. We will generate a pair of keys for you.
 +
 +
    Created: /root/.ssh/id_rsa.pub
 +
 +
Your public SSH key must be uploaded to the OpenShift server to access code.  Upload
 +
now? (yes|no)
 +
yes
 +
 +
Since you do not have any keys associated with your OpenShift account, your new key
 +
will be uploaded as the 'default' key.
 +
 +
Uploading key 'default' ... done
 +
 +
Checking for git ... found git version 1.8.2.1
 +
 +
Checking common problems .. done
 +
 +
Checking your namespace ... none
 +
 +
Your namespace is unique to your account and is the suffix of the public URLs we
 +
assign to your applications. You may configure your namespace here or leave it blank
 +
and use 'rhc create-domain' to create a namespace later.  You will not be able to
 +
create applications without first creating a namespace.
 +
 +
Please enter a namespace (letters and numbers only) |<none>|: demoland
 +
 +
Create an app
 +
rhc domain show -p demo
 +
rhc app create test1 diy-0.1 -p demo
 +
 +
==Test on Local Machine (after node is back up)==
 +
Setup your machine to use broker as a name server (Note: This might mess up normal network operations.)
 +
vi /etc/resolve.conf
 +
# At the first line put "nameserver *broker ip address*"
 +
nameserver 192.168.122.220
 +
 +
'''Check and Setup User'''
 +
yum -y install rubygem-rhc
 +
LIBRA_SERVER=broker.example.com rhc setup
 +
Should look like (Note: Generate a token now? no - client can handle it, broker in F19 cannot)
 +
OpenShift Client Tools (RHC) Setup Wizard
 +
 +
This wizard will help you upload your SSH keys, set your application namespace, and
 +
check that other programs like Git are properly installed.
 +
 +
The server's certificate is self-signed, which means that a secure connection can't be
 +
established to 'broker.example.com'.
 +
 +
You may bypass this check, but any data you send to the server could be intercepted by
 +
others.
 +
 +
Connect without checking the certificate? (yes|no): yes
 +
Login to broker.example.com: demo
 +
Password: ****
 +
 
 +
OpenShift can create and store a token on disk which allows to you to access the
 +
server without using your password. The key is stored in your home directory and
 +
should be kept secret.  You can delete the key at any time by running 'rhc logout'.
 +
Generate a token now? (yes|no) no
 +
 +
Saving configuration to /root/.openshift/express.conf ... done
 +
 +
No SSH keys were found. We will generate a pair of keys for you.
 +
 +
    Created: /root/.ssh/id_rsa.pub
 +
 +
Your public SSH key must be uploaded to the OpenShift server to access code.  Upload
 +
now? (yes|no)
 +
yes
 +
 +
Since you do not have any keys associated with your OpenShift account, your new key
 +
will be uploaded as the 'default' key.
 +
 +
Uploading key 'default' ... done
 +
 +
Checking for git ... found git version 1.8.2.1
 +
 +
Checking common problems .. done
 +
 +
Checking your namespace ... none
 +
 +
Your namespace is unique to your account and is the suffix of the public URLs we
 +
assign to your applications. You may configure your namespace here or leave it blank
 +
and use 'rhc create-domain' to create a namespace later.  You will not be able to
 +
create applications without first creating a namespace.
 +
 +
Please enter a namespace (letters and numbers only) |<none>|: demoland
 +
 +
Create an app
 +
rhc domain show -p demo
 +
rhc app create test2 diy-0.1 -p demo
 +
 +
 +
'''Check App'''
 +
You should be able to go to the following URL in your web browser.
 +
 +
http://test2-demoland.example.com/

Latest revision as of 13:39, 8 August 2013

Fedora 19 is when OpenShift Origin first became a feature.

NOTE: (August 8, 2013) This page is getting an update. It will accommodate F19 cloud images (not just minimal install). It is also updated with the OpenShift Origin Version 2 documentation.

This page is here to show how to setup OpenShift Origin on Fedora 19 using the packages in Fedora, as opposed to the packages published from upstream. These steps are written out to be done by hand. Yes, people can script and/or puppetize these steps. But these are written out so that people can see, and fine tune them.

Note: And now they have been written into scripts. https://github.com/tdawson/oo-install-scripts

Goal: By the end of this, you should have two machines. A broker machine, and one node machine. You should be able to create applications, that will be put on the node machine. You should be able to check the status of those applications. You should be able to point your web browser to the URL of those applications.

Note: There is no web console in Fedora 19. That will be in Fedora 20.

These instructions were created most from the following two places.

Contents

[edit] Initial Setup of Broker and Node Machines

ON BOTH BROKER AND NODE

# Start with a Fedora 19 minimal install
yum -y update
# avoid clock skew
yum -y install ntp
/bin/systemctl enable ntpd.service
/bin/systemctl start  ntpd.service

ON BROKER

export DOMAIN="example.com"
export BROKERIP="$(nm-tool | grep Address | grep -v HW | awk '{print $2}')"
export BROKERNAME="broker.example.com"
export NODEIP="--- IP Address from Node machine ---"
export NODENAME="node.example.com"
# Here is the IP Address from Broker machine
nm-tool | grep Address | grep -v HW | awk '{print $2}'

ON NODE

export DOMAIN="example.com"
export BROKERIP="--- IP Address from Broker machine ---"
export BROKERNAME="broker.example.com"
export NODEIP="$(nm-tool | grep Address | grep -v HW | awk '{print $2}')"
export NODENAME="node.example.com"
# Here is the IP Address from Node machine
nm-tool | grep Address | grep -v HW | awk '{print $2}'

[edit] Setup and Configure Broker

[edit] Broker: Bind DNS

yum -y install bind bind-utils

KEYFILE=/var/named/${DOMAIN}.key

setup DNSSEC key pair

cd /var/named/
dnssec-keygen -a HMAC-MD5 -b 512 -n USER -r /dev/urandom ${DOMAIN}
KEY="$(grep Key: K${DOMAIN}*.private | cut -d ' ' -f 2)"
cd -
rndc-confgen -a -r /dev/urandom
echo $KEY

setup permissions for the DNSSEC key pair

restorecon -v /etc/rndc.* /etc/named.*
chown -v root:named /etc/rndc.key
chmod -v 640 /etc/rndc.key

setup forwarders

echo "forwarders { 8.8.8.8; 8.8.4.4; } ;" >> /var/named/forwarders.conf
restorecon -v /var/named/forwarders.conf
chmod -v 755 /var/named/forwarders.conf

setup initial DNS database

rm -rvf /var/named/dynamic
mkdir -vp /var/named/dynamic

cat <<EOF > /var/named/dynamic/${DOMAIN}.db
\$ORIGIN .
\$TTL 1	; 1 seconds (for testing only)
${DOMAIN} IN SOA ns1.${DOMAIN}. hostmaster.${DOMAIN}. (
                         2011112904 ; serial
                         60         ; refresh (1 minute)
                         15         ; retry (15 seconds)
                         1800       ; expire (30 minutes)
                         10         ; minimum (10 seconds)
                          )
                     NS ns1.${DOMAIN}.
                     MX 10 mail.${DOMAIN}.
\$ORIGIN ${DOMAIN}.
ns1	              A        127.0.0.1

EOF

Install the DNSSEC key

cat <<EOF > ${KEYFILE}
key ${DOMAIN} {
  algorithm HMAC-MD5;
  secret "${KEY}";
};
EOF

Check the key and database

cat /var/named/dynamic/${DOMAIN}.db
cat /var/named/${DOMAIN}.key

Set permissions for key and database

chown -Rv named:named /var/named
restorecon -rv /var/named

Create the named configuration file

mv /etc/named.conf /etc/named.conf.openshift
cat <<EOF > /etc/named.conf
// named.conf
//
// Provided by Red Hat bind package to configure the ISC BIND named(8) DNS
// server as a caching only nameserver (as a localhost DNS resolver only).
//
// See /usr/share/doc/bind*/sample/ for example named configuration files.
//

options {
    listen-on port 53 { any; };
    directory "/var/named";
    dump-file "/var/named/data/cache_dump.db";
    statistics-file "/var/named/data/named_stats.txt";
    memstatistics-file "/var/named/data/named_mem_stats.txt";
    allow-query { any; };
    recursion yes;

    /* Path to ISC DLV key */
    bindkeys-file "/etc/named.iscdlv.key";

    // set forwarding to the next nearest server (from DHCP response)
    forward only;
    include "forwarders.conf";
};

logging {
    channel default_debug {
        file "data/named.run";
        severity dynamic;
    };
};

// use the default rndc key
include "/etc/rndc.key";
 
controls {
    inet 127.0.0.1 port 953
    allow { 127.0.0.1; } keys { "rndc-key"; };
};

include "/etc/named.rfc1912.zones";

include "${DOMAIN}.key";

zone "${DOMAIN}" IN {
    type master;
    file "dynamic/${DOMAIN}.db";
    allow-update { key ${DOMAIN} ; } ;
};
EOF

Check the named config file

cat /etc/named.conf

setup permissions of named config file

chown -v root:named /etc/named.conf
restorecon /etc/named.conf

Setup firewall

firewall-cmd --add-service=dns
firewall-cmd --permanent --add-service=dns
firewall-cmd --list-all

Setup and start service

/bin/systemctl enable named.service
/bin/systemctl start named.service

add entries using nsupdate

nsupdate -k ${KEYFILE}
> server 127.0.0.1
> update delete broker.example.com A
> update add **your broker full name ** 180 A **your broker ip address**
(example: update add broker.example.com 180 A 192.168.122.220 )
> send
> quit

Test DNS server This is best done before hostname has been set.

ping broker.example.com
dig @127.0.0.1 broker.example.com

[edit] Broker: DHCP client and hostname

Setup dhcp client

echo "prepend domain-name-servers **your broker ip address**;" >> /etc/dhcp/dhclient-eth0.conf
echo "supersede host-name \"broker\";" >> /etc/dhcp/dhclient-eth0.conf
echo "supersede domain-name \"example.com\";" >> /etc/dhcp/dhclient-eth0.conf

Setup hostname

echo "broker.example.com" > /etc/hostname

[edit] Broker: MongoDB

Install Software

yum -y install mongodb-server

Tweak config file

vi /etc/mongodb.conf
# Uncomment auth = true
# Add smallfiles = true

Setup and start service

/usr/bin/systemctl enable mongod.service
/usr/bin/systemctl status mongod.service
/usr/bin/systemctl start mongod.service
/usr/bin/systemctl status mongod.service

Testing

mongo
> show dbs
> exit

[edit] Broker: Messaging (using QPID)

Activemq on F19 isn't ready for OpenShift production. When it is, we'll use that For now we'll use QPID with mcollective.

Install Software

yum install mcollective-qpid-plugin qpid-cpp-server

Setup Firewall

firewall-cmd --add-port=5672/tcp
firewall-cmd --permanent --add-port=5672/tcp
firewall-cmd --list-all

Setup and start service

/usr/bin/systemctl enable qpidd.service
/usr/bin/systemctl start qpidd.service
/usr/bin/systemctl status qpidd.service

[edit] Broker: MCollective client ( using QPID)

Install Software

yum -y install mcollective-client

Move original config file out of the way

mv /etc/mcollective/client.cfg /etc/mcollective/client.cfg.orig

Create new client config file. This config file is for using QPID as a messaging platform.

cat <<EOF > /etc/mcollective/client.cfg
topicprefix = /topic/
main_collective = mcollective
collectives = mcollective
libdir = /usr/libexec/mcollective
loglevel = debug
logfile = /var/log/mcollective-client.log

# Plugins
securityprovider = psk
plugin.psk = unset
connector = qpid
plugin.qpid.host=${BROKERNAME}
plugin.qpid.secure=false
plugin.qpid.timeout=5

# Facts
factsource = yaml
plugin.yaml = /etc/mcollective/facts.yaml
EOF

[edit] Broker: broker application

Install software

yum -y install openshift-origin-broker openshift-origin-broker-util rubygem-openshift-origin-auth-remote-user rubygem-openshift-origin-msg-broker-mcollective rubygem-openshift-origin-dns-bind

Modify the broker proxy server name

sed -i -e "s/ServerName .*$/ServerName broker.example.com/" /etc/httpd/conf.d/000002_openshift_origin_broker_servername.conf 
cat /etc/httpd/conf.d/000002_openshift_origin_broker_servername.conf

Setup and start service

/usr/bin/systemctl enable httpd.service
/usr/bin/systemctl enable ntpd.service
/usr/bin/systemctl enable sshd.service

Setup Firewall

firewall-cmd --add-service=ssh
firewall-cmd --add-service=http
firewall-cmd --add-service=https
firewall-cmd --permanent --add-service=ssh
firewall-cmd --permanent --add-service=http
firewall-cmd --permanent --add-service=https
firewall-cmd --list-all

Generate access key

openssl genrsa -out /etc/openshift/server_priv.pem 2048
openssl rsa -in /etc/openshift/server_priv.pem -pubout > /etc/openshift/server_pub.pem
ssh-keygen -t rsa -b 2048 -f ~/.ssh/rsync_id_rsa
cp -v ~/.ssh/rsync_id_rsa* /etc/openshift/

Setup selinux boolean variables and set file contexts

setsebool -P httpd_unified=on httpd_can_network_connect=on httpd_can_network_relay=on httpd_run_stickshift=on named_write_master_zones=on
fixfiles -R rubygem-passenger restore
fixfiles -R mod_passenger restore
restorecon -rv /var/run
restorecon -rv /usr/share/gems/gems/passenger-*

Tweak broker config, if needed

vi /etc/openshift/broker.conf
# Might not have to do anything but make sure you have the following lines
CLOUD_DOMAIN="example.com"
VALID_GEAR_SIZES="small,medium"

[edit] Broker: broker plugins and MongoDB user accounts

Create config files from examples

cp /etc/openshift/plugins.d/openshift-origin-msg-broker-mcollective.conf.example /etc/openshift/plugins.d/openshift-origin-msg-broker-mcollective.conf

Config the DNS plugin

cd /var/named/
KEY="$(grep Key: K${DOMAIN}*.private | cut -d ' ' -f 2)"
cat $KEYFILE
echo $KEY

cat <<EOF > /etc/openshift/plugins.d/openshift-origin-dns-bind.conf
BIND_SERVER="127.0.0.1"
BIND_PORT=53
BIND_KEYNAME="${DOMAIN}"
BIND_KEYVALUE="${KEY}"
BIND_ZONE="${DOMAIN}"
EOF

Configure authentication plugin and add a user

cp -v /var/www/openshift/broker/httpd/conf.d/openshift-origin-auth-remote-user-basic.conf.sample /var/www/openshift/broker/httpd/conf.d/openshift-origin-auth-remote-user.conf
htpasswd -c -b -s /etc/openshift/htpasswd demo demo
# Don't forget your password. <demo password>
cat /etc/openshift/htpasswd

Add MongoDB account

grep MONGO /etc/openshift/broker.conf
mongo openshift_broker_dev --eval 'db.addUser("openshift", "mooo")'
# If you are going to change the username and/or password, change broker.conf

Bundle broker gems

yum -y install rubygem-psych rubygem-mocha
cd /var/www/openshift/broker
gem install mongoid
bundle --local

Setup and start services

/usr/bin/systemctl enable openshift-broker.service
/usr/bin/systemctl start httpd.service
/usr/bin/systemctl start openshift-broker.service
/usr/bin/systemctl status openshift-broker.service

Test basic broker service

curl -k -u demo:demo https://localhost/broker/rest/api

[edit] Setup and Configure Node

[edit] Node: Initial setup/configure

ON BROKER

KEYFILE=/var/named/${DOMAIN}.key

Register the node in DNS

oo-register-dns -h ${NODENAME} -d ${DOMAIN} -n ${NODEIP} -k ${KEYFILE}

Copy the broker public key to node

scp /etc/openshift/rsync_id_rsa.pub root@${NODENAME}:/root/.ssh/

ON NODE Put the brokers public key in root authorized keys

cat /root/.ssh/rsync_id_rsa.pub >> /root/.ssh/authorized_keys
rm -f /root/.ssh/rsync_id_rsa.pub

ON BROKER Test to make sure we can login using our key

ssh -i /root/.ssh/rsync_id_rsa root@${NODENAME}
exit

[edit] Node: DHCP client and hostname

Configure the dhcp settings

echo "prepend domain-name-servers **your broker ip address**;" >> /etc/dhcp/dhclient-eth0.conf
echo "supersede host-name \"node\";" >> /etc/dhcp/dhclient-eth0.conf
echo "supersede domain-name \"example.com\";" >> /etc/dhcp/dhclient-eth0.conf

Set the hostname

echo "node.example.com" > /etc/hostname

[edit] Node: MCollective

ON NODE Install Software

yum -y install openshift-origin-msg-node-mcollective mcollective-qpid-plugin

Move original configuration out of the way

mv /etc/mcollective/server.cfg /etc/mcollective/server.cfg.orig

Create new configuration

cat <<EOF > /etc/mcollective/server.cfg
topicprefix = /topic/
main_collective = mcollective
collectives = mcollective
libdir = /usr/libexec/mcollective
logfile = /var/log/mcollective.log
loglevel = debug
daemonize = 1
direct_addressing = n

# Plugins
securityprovider = psk
plugin.psk = unset
connector = qpid
plugin.qpid.host=${BROKERNAME}
plugin.qpid.secure=false
plugin.qpid.timeout=5

# Facts
factsource = yaml
plugin.yaml = /etc/mcollective/facts.yaml
EOF

Setup and start services

/bin/systemctl enable mcollective.service
/bin/systemctl start  mcollective.service

ON BROKER

mco ping
# node should show up on mco ping

[edit] Node: node application

Install software

yum -y install rubygem-openshift-origin-node rubygem-passenger-native openshift-origin-port-proxy openshift-origin-node-util
yum -y install openshift-origin-cartridge-cron-1.4 openshift-origin-cartridge-diy-0.1

Setup firewall

firewall-cmd --add-service=ssh
firewall-cmd --add-service=http
firewall-cmd --add-service=https
firewall-cmd --permanent --add-service=ssh
firewall-cmd --permanent --add-service=http
firewall-cmd --permanent --add-service=https
firewall-cmd --list-all

[edit] Node: PAM namespace module, cgroups, and user quotas

PAM

sed -i -e 's|pam_selinux|pam_openshift|g' /etc/pam.d/sshd

for f in "runuser" "runuser-l" "sshd" "su" "system-auth-ac"
do
  t="/etc/pam.d/$f"
  if ! grep -q "pam_namespace.so" "$t"
  then
    echo -e "session\t\trequired\tpam_namespace.so no_unmount_on_close" >> "$t"
  fi
done

CGROUPS

Cgroups Config - Need to still fixup the cgroup configurations

echo "mount {" >> /etc/cgconfig.conf
echo "        cpu     = /cgroup/all;" >> /etc/cgconfig.conf
echo "        cpuacct = /cgroup/all;" >> /etc/cgconfig.conf
echo "        memory  = /cgroup/all;" >> /etc/cgconfig.conf
echo "        freezer = /cgroup/all;" >> /etc/cgconfig.conf
echo "        net_cls = /cgroup/all;" >> /etc/cgconfig.conf
echo "}" >> /etc/cgconfig.conf
restorecon -v /etc/cgconfig.conf
mkdir /cgroup
restorecon -RFvv /cgroup

Cgroups enable and startup services

/bin/systemctl enable cgconfig.service
/bin/systemctl enable cgred.service
/usr/sbin/chkconfig openshift-cgroups on
/bin/systemctl restart  cgconfig.service
/bin/systemctl restart  cgred.service
/usr/sbin/service openshift-cgroups restart

DISK QUOTA

# Edit fstab and add usrquota to whichever filesystem 
#   has /var/lib/openshift on it
UUID=b9e21eae-4b8c-4936-9f5d-d10631ff535e / ext4    defaults,usrquota 1 1
# reboot or remount
mount -o remount /
quotacheck -cmug /

[edit] Node: SELinux and System Control

Setup SELINUX Booleans

setsebool -P httpd_unified=on httpd_can_network_connect=on httpd_can_network_relay=on httpd_read_user_content=on httpd_enable_homedirs=on httpd_run_stickshift=on allow_polyinstantiation=on

Update selinux file setting

restorecon -rv /var/run
restorecon -rv /usr/sbin/mcollectived /var/log/mcollective.log /var/run/mcollectived.pid
restorecon -rv /var/lib/openshift /etc/openshift/node.conf /etc/httpd/conf.d/openshift

SYSTEM CONTROL SETTINGS

echo "# Added for OpenShift" >> /etc/sysctl.d/openshift.conf
echo "kernel.sem = 250  32000 32  4096" >> /etc/sysctl.d/openshift.conf
echo "net.ipv4.ip_local_port_range = 15000 35530" >> /etc/sysctl.d/openshift.conf
echo "net.netfilter.nf_conntrack_max = 1048576" >> /etc/sysctl.d/openshift.conf
sysctl -p /etc/sysctl.d/openshift.conf

[edit] Node: SSH, Port Proxy, and Node application

SSH

vi /etc/ssh/sshd_config
> AcceptEnv GIT_SSH

perl -p -i -e "s/^#MaxSessions .*$/MaxSessions 40/" /etc/ssh/sshd_config
perl -p -i -e "s/^#MaxStartups .*$/MaxStartups 40/" /etc/ssh/sshd_config

/bin/systemctl restart  sshd.service

PORT PROXY

firewall-cmd --add-port=35531-65535/tcp
firewall-cmd --permanent --add-port=35531-65535/tcp
firewall-cmd --list-all

/bin/systemctl enable openshift-port-proxy.service
/bin/systemctl restart  openshift-port-proxy.service

NODE SETUP

/bin/systemctl enable httpd.service
/bin/systemctl enable openshift-gears.service

vi /etc/openshift/node.conf
> PUBLIC_HOSTNAME="node.example.com"
> PUBLIC_IP="192.168.122.161" (Node IP Address)
> BROKER_HOST="192.168.122.220" (Broker IP Address)
> CLOUD_DOMAIN="example.com"

/etc/cron.minutely/openshift-facts

[edit] Node: Reboot

We need to reboot to load all the node stuff correctly

reboot

[edit] Testing

[edit] Test on Broker (after node is back up)

Check Messaging

mco ping

Should look like

node.example.com                         time=239.51 ms

---- ping statistics ----
1 replies max: 239.51 min: 239.51 avg: 239.51 

Check Broker

curl -k -u demo:demo https://localhost/broker/rest/api

Should look like

{"data":{"API":{"href":"https://localhost/broker/rest/api","method":"GET","optional_params":[],"rel":"API entry point","required_params":[]},"GET_ENVIRONMENT":{"href":"https://localhost/broker/rest/environment","method":"GET","optional_params":[],"rel":"Get environment information","required_params":[]},"GET_USER"
...
:id","type":"string","valid_options":[]}]}},"messages":[],"status":"ok","supported_api_versions":[1.0,1.1,1.2,1.3],"type":"links","version":"1.3"}

Check and Setup User

yum -y install rubygem-rhc
LIBRA_SERVER=broker.example.com rhc setup

Should look like (Note: Generate a token now? no - client can handle it, broker in F19 cannot)

OpenShift Client Tools (RHC) Setup Wizard

This wizard will help you upload your SSH keys, set your application namespace, and
check that other programs like Git are properly installed.

The server's certificate is self-signed, which means that a secure connection can't be
established to 'broker.example.com'.

You may bypass this check, but any data you send to the server could be intercepted by
others.

Connect without checking the certificate? (yes|no): yes
Login to broker.example.com: demo
Password: ****
 
OpenShift can create and store a token on disk which allows to you to access the
server without using your password. The key is stored in your home directory and
should be kept secret.  You can delete the key at any time by running 'rhc logout'.
Generate a token now? (yes|no) no

Saving configuration to /root/.openshift/express.conf ... done

No SSH keys were found. We will generate a pair of keys for you.

    Created: /root/.ssh/id_rsa.pub

Your public SSH key must be uploaded to the OpenShift server to access code.  Upload
now? (yes|no)
yes

Since you do not have any keys associated with your OpenShift account, your new key
will be uploaded as the 'default' key.

Uploading key 'default' ... done

Checking for git ... found git version 1.8.2.1

Checking common problems .. done

Checking your namespace ... none

Your namespace is unique to your account and is the suffix of the public URLs we
assign to your applications. You may configure your namespace here or leave it blank
and use 'rhc create-domain' to create a namespace later.  You will not be able to
create applications without first creating a namespace.

Please enter a namespace (letters and numbers only) |<none>|: demoland

Create an app

rhc domain show -p demo
rhc app create test1 diy-0.1 -p demo

[edit] Test on Local Machine (after node is back up)

Setup your machine to use broker as a name server (Note: This might mess up normal network operations.)

vi /etc/resolve.conf
# At the first line put "nameserver *broker ip address*"
nameserver 192.168.122.220

Check and Setup User

yum -y install rubygem-rhc
LIBRA_SERVER=broker.example.com rhc setup

Should look like (Note: Generate a token now? no - client can handle it, broker in F19 cannot)

OpenShift Client Tools (RHC) Setup Wizard

This wizard will help you upload your SSH keys, set your application namespace, and
check that other programs like Git are properly installed.

The server's certificate is self-signed, which means that a secure connection can't be
established to 'broker.example.com'.

You may bypass this check, but any data you send to the server could be intercepted by
others.

Connect without checking the certificate? (yes|no): yes
Login to broker.example.com: demo
Password: ****
 
OpenShift can create and store a token on disk which allows to you to access the
server without using your password. The key is stored in your home directory and
should be kept secret.  You can delete the key at any time by running 'rhc logout'.
Generate a token now? (yes|no) no

Saving configuration to /root/.openshift/express.conf ... done

No SSH keys were found. We will generate a pair of keys for you.

    Created: /root/.ssh/id_rsa.pub

Your public SSH key must be uploaded to the OpenShift server to access code.  Upload
now? (yes|no)
yes

Since you do not have any keys associated with your OpenShift account, your new key
will be uploaded as the 'default' key.

Uploading key 'default' ... done

Checking for git ... found git version 1.8.2.1

Checking common problems .. done

Checking your namespace ... none

Your namespace is unique to your account and is the suffix of the public URLs we
assign to your applications. You may configure your namespace here or leave it blank
and use 'rhc create-domain' to create a namespace later.  You will not be able to
create applications without first creating a namespace.

Please enter a namespace (letters and numbers only) |<none>|: demoland

Create an app

rhc domain show -p demo
rhc app create test2 diy-0.1 -p demo


Check App You should be able to go to the following URL in your web browser.

http://test2-demoland.example.com/