Netapp Infrastructure SOP
Provides primary mirrors and additional storage in PHX
Owner: Fedora Infrastructure Team
Contact: #fedora-admin, sysadmin-main, releng
Location: Phoenix, Tampa Bay, Raleigh
Servers: puppet1, internal, xen servers, application servers, builders, releng boxes
Purpose: Provides primary mirrors and additional storage in PHX
At present we have three netapps in our infrastructure. One in TPA, RDU and PHX. For purposes of visualization its easiest to think of us as having 4 netapps, 1 TPA, 1 RDU and 1 PHX for public mirrors. And an additional 1 in PHX used for additional storage not related to the public mirrors.
The netapps are our primary public mirrors. The canonical location for the mirrors is currently in PHX. From there it gets synced to RDU and TPA.
Snapshots on the PHX netapp are taken hourly. Unfortunately the way it is setup only Red Hat employees can access this mirror (this is scheduled to change when PHX becomes the canonical location but that will take time to setup and deploy). The snapshots are available, for example, on wallace in:
PHX NFS Storage
There is a great deal of storage in PHX over NFS from the netapp there. This storage includes the public mirror. The majority of this storage is koji however there are a few gig worth of storage that goes to wiki attachments and other storage needs we have in PHX.
You can access all of the nfs share shares at:
puppet1:/mnt/fedora or ntap-fedora1.fedora.phx.redhat.com:/vol/fedora/
The netapp is provided by RHIS and as a result they also control access. Access is controlled by IP mostly and some machines have root squashed. Worst case scenario if puppet1 is not accessible, just bring another box up under its IP address and use that for an emergency.
There are hourly and nightly snapshots on the netapp. They are available in:
We have iscsi deployed in a number of locations in our infrastructure for xen machines. To get a list of what xen machines are deployed with iscsi, just run lvs:
Live migration is possible though not fully supported at this time. Please shut a xen machine down and bring it up on another host. Memory is the main issue here.
iscsi is mounted all over the place and if one xen machine creates a logical volume the other xen machines will have to pick up those changes. To do this run:
pvscan vgscan lvscan vgchange -a y
On reboots sometimes the iscsi share is not remounted. This should be automated in the future but for now run:
iscsiadm -m discovery -tst -p ntap-fedora1.fedora.phx.redhat.com:3260 sleep 1 iscsiadm -m node -T iqn.1992-08.com.netapp:sn.101197194 -p 10.8.34.17:3260 -l sleep 1 pvscan vgscan lvscan vgchange -a y