Staging Servers Infrastructure SOP
We have the ability to create a staging environment in Infrastructure. This attempts to duplicate a small section of our infrastructure for testing purposes. If there's a large scale change that you need to perform, it's a good idea to push the changes to the staging environment first and check that everything works properly and that you have puppetized all the necessary changes before pushing the changes live.
Owner: Fedora Infrastructure Team
Servers: (proxy1.stg, db1.stg, app1.stg, app2.stg fas1.stg)
Purpose: Staging environment to test changes to apps.
Staging servers are managed through puppet so that you can test config changes in puppet before pushing live. You need to use a different branch to push to the staging servers. Basic steps:
git clone /git/puppet/ git checkout --track -b staging origin/staging # This only needs to be run once on the initial clone # make changes, git commits git push git checkout master # switch back to the master branch git checkout staging # switch back to the staging branch
Sometimes you want to resync the staging fas server with what's on production. To do that, dump what's in the production db and then import it into the staging db. Note that resyncing the information will remove any of the information that has been added to the staging fas servers. So it's good to mention that you're doing this on the infra list or to people who you know are working on the staging fas servers so they can either save their changes or ask you to hold off for a while.
On db02: $ ssh db02 $ sudo -u postgres pg_dump -C fas2 |xz -c fas2.dump.xz $ scp fas2.dump.xz db01.stg: On fas01.stg (postgres won't drop the database if something is accessing it) (ATM, fas in staging is not load balanced so we only have to do this on one server) $ sudo /etc/init.d/httpd stop On db01.stg: $ sudo -u postgres psql 'drop database fas2;' $ xzcat fas2.dump.xz | sudo -u postgres psql On fas01.stg: $ sudo /etc/init.d/httpd start