Clustered Samba for GFS2
Support for clustered samba (including active/active configuration) over GFS2 using CTDB.
- Name: Abhijith Das
- email: email@example.com
- Targeted release: Fedora 12
- Last updated: 9/29/2009
- Percentage of completion: 100%
- I've been able to test the following:
- Simultaneous samba exports of the same share from multiple cluster nodes (active/active samba)
- Export the shares through a public IP address and mount from a client node using this public IP address
- Run successful smbtorture nbench tests from a client
- Active failover of samba service from one node (when killed) to another. I've _not_ tested this in f12 as I encountered a gfs2 recovery hang (it's a gfs2 bug I'm working on, has nothing to do with clustered samba), but I have seen this work with RHEL5 testing of gfs2-clustered-samba and should work in f12 as well.
With the availability of a clustered database (CTDB) for Samba, multiple instances of smbd across different nodes of a cluster are able to share state. It is now possible to export a gfs2 filesystem through one or more nodes in the cluster in a active/passive or active/active configuration to provide a highly-available Samba service.
Benefit to Fedora
Samba was not cluster-aware until the introduction of CTDB and hence it was not possible to have a highly available samba service. Particularly, active/active configuration was not possible. With this feature, it will be possible to run highly available samba service(s) with different configurations (active/active, active/passive) over a shared gfs2 filesystem on the Red Hat cluster infrastructure.
- Verify simple functionality by exporting samba shares over gfs2 and running smbtorture tests (in progress)
- fcntl posix locks performance: Improve upon gfs2's posix lock performance (single-node & clustered) as smbd uses them extensively when posix locking is turned on. (in progress)
- Get CTDB and CTDB-supported Samba packages into F12. (complete)
- Get Conga interface ready for manipulating the various config files (/etc/cluster/cluster.conf, /etc/sysconfig/ctdb, /etc/samba/smb.conf) through web-based GUI. (Alternative to hand-editing configs) (in progress)
- Verify working of clustered samba use cases such as service failover, recovery, etc., and document respective configuration parameters. (in progress)
This feature will not affect the operation of non-clustered samba setups or other gfs2-based setups. It will provide the capability to setup clustered samba over gfs2.
How To Test
- cluster nodes (2 or more) connected to shared storage
- client node(s) to run client loads from.
- kernel with gfs2 and dlm enabled
- Red Hat cluster packages cman/openais/corosync/lvm2-cluster etc. See usage.txt for more information.
- Latest samba and ctdb packages.
- Setup the cluster.conf file and start up the cluster 'service cman start'
- If you're using clustered lvm, 'service clvmd start'
- mount a gfs2-formatted clustered volume at the same mountpoint on all nodes.
- Export this mountpoint (or a directory within it) as a samba share. All smb.conf files in all nodes should be identical
- Set ctdb configuration. See ctdb setup
- start the ctdb service and check 'ctdb status' to make sure the ctdb cluster is up.
- Now you should simultaneously be able to access the samba share through all the exporting nodes and run operations on them simultaneously.
- The smbtorture test suite in the samba package has a few scenarios RAW_BENCH_LOCK, RAW_BENCH_OPEN, BENCH_NBENCH etc. See testing ctdb
- There's also the ping_pong test at: ping_pong doc
- Any other test you can think of with multiple clients (windows users mounting the samba share) that stresses ctdb/gfs2.
Users not using gfs2, samba or ctdb should notice no change. Users will now have the ability to run a clustered samba setup exporting gfs2 shares.
Most of the kernel support (gfs2 filesystem) is already upstream. ctdb and samba packages have been included in Fedora.
In it's current state, a basic clustered samba setup should work. Enhancements such as posix lock performance & conga interface and testing of different use-cases would be nice to get done with. If not done with these tasks, in it's current form, the feature is still acceptable.
- Cluster Suite on Wikipedia
- GFS/GFS2 on Wikipedia
- Cluster Wiki page
- GFS2 kernel documentation (a very basic introduction)
- Fedora now has support for clustered samba over gfs2
Comments and Discussion