From Fedora Project Wiki
(27 intermediate revisions by 3 users not shown)
Line 1: Line 1:
= Cluster =
= Updated Cluster Stack with Enhanced Failover Support =


== Summary ==
== Summary ==


Update the corosync/openais/cluster/clvm stack to the latest stable releases.
This is a significant update to the clustering stack - both for high availability and load balancing.
The new upstream versions brings a new and more powerful cluster stack to the end user.


== Owner ==
== Owner ==
* Name: [[User:fabbione| FabioMassimoDiNitto]]
* email: fdinitto@redhat.com
* Name: [[User:lon|LonHohberger]]
* Name: [[User:lon|LonHohberger]]
* email: lhh@redhat.com
* email: lhh@redhat.com
* Name: [[User:fabbione| FabioMassimoDiNitto]]
* email: fabbione@redhat.com


== Current status ==
== Current status ==
* Targeted release: [[Releases/{{FedoraVersion||next}} | {{FedoraVersion|long|next}} ]]  
* Targeted release: [[Releases/{{FedoraVersion||next}} | {{FedoraVersion|long|next}} ]]  
* Last updated: 9 Jul 2009
* Last updated: 26 Mar 2012
* Percentage of completion: 99%
* Percentage of completion: 100%


== Detailed Description ==
== Detailed Description ==
Upstream has been developing the new version of the stack with several goals in mind, including better scalability, higher performance, increased reliability, more hardware support.  The updated release lays down the path of better integration with other cluster solutions (such as [http://www.linux-ha.org heartbeat], [http://oss.oracle.com/projects/ocfs2/ OCFS2], and [http://www.clusterlabs.org pacemaker]) by sharing the same core components.
Upstream has been developing a new version of the cluster stack which is currently shipped with Fedora 16.  The major features are:
* Improved quorum subsystem which is integrated in to the Corosync Cluster Engine (95% complete)
* A new command-line interface for administration of both the Corosync Cluster Engine and the Pacemaker Cluster Resource Manager as well as monitoring (50% complete)
* Convergence on a single set of resource agents from the Linux clustering community
* Enhanced fencing support provided by stonith-ng, which is part of the Pacemaker project.
* Separation of GFS2-specific utilities in to a separate project (100%)
* Separation of DLM-specific utilities in to a separate project (90%)
 
Note that this transition includes deprecations:
* The cluster package, including CMAN, DLM utilities, GFS2 utilities, fenced, and all associated libraries.
* The rgmanager package.  All users are advised to use Pacemaker for their failover needs.
* The openais packageThis provided some of the SA Forum AIS APIs, which were rarely used.
* The heartbeat package. Upstream development has stopped and equivalent or superior functionality is provided by the [http://corosync.org Corosync] package.
* The piranha package.  Superior functionality is provided by the [http://keepalived.org Keepalived] package.
* The ricci package.  Similar agent-style communication can be provided by Matahari or other projects.
* The luci package.  This will be replaced by Sunzi, although it is not clear this part will make Fedora 17.
* The clustermon package.  This was part of the ricci/luci administration stack and is no longer required.
* Several agents in the resource-agents package which previously relied on examination of cluster.conf at run-time.


== Benefit to Fedora ==
== Benefit to Fedora ==
The increased reliability and versatility of the cluster components included in Fedora 12 allow administrators to deploy Fedora in environments where greater availability and clustered file systems are required.
The increased reliability and versatility of the cluster components included in Fedora 17 allow administrators to deploy Fedora in environments where greater availability and clustered file systems are required.
<!-- What is the benefit to the platform?  If this is a major capability update, what has changed?  If this is a new feature, what capabilities does it bring? Why will Fedora become a better distribution or project because of this feature?-->
<!-- What is the benefit to the platform?  If this is a major capability update, what has changed?  If this is a new feature, what capabilities does it bring? Why will Fedora become a better distribution or project because of this feature?-->
Removal of rgmanager matches the capabilities offered by several other Linux distributions and allows the Fedora community to consort efforts on a single failover stack.


== Scope ==
== Scope ==
<!-- What work do the developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
<!-- What work do the developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
This change impacts several utilities and other applications, including:
* The DLM, which used AIS Checkpoints to replicate data about POSIX locks (100% complete)
* Clustered LVM, which used CMAN APIs, will need to be ported to corosync. (?% complete)
* Fence-virt, which used CMAN APIs and AIS Checkpoints, will need to be ported to corosync. (0% complete)
* Pacemaker, which used CMAN APIs, will need to be ported to corosync (100% complete)
* Clustered mirroring, which uses AIS Checkpoints will need to be ported to corosync (?% complete)
* QPid, which used CMAN APIs, will need to be ported to corosync (?% complete)


== How To Test ==
== How To Test ==
Line 41: Line 65:
3. What are the expected results of those actions?
3. What are the expected results of those actions?
-->
-->
The initial use cases required for high availability are documented in the [http://www.clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Clusters_from_Scratch/ Clusters from Scratch] manual.
Information about how to run GFS2 in standalone mode will be provided as development nears completion.


== User Experience ==
== User Experience ==
<!-- If this feature is noticeable by its target audience, how will their experiences change as a result?  Describe what they will see or notice. -->
<!-- If this feature is noticeable by its target audience, how will their experiences change as a result?  Describe what they will see or notice. -->
Some differences will be noticeable by users:
* Users of the rgmanager package will need to learn the Pacemaker package, which is a more dynamic, complex, full-featured cluster resource manager.
* Users who relied on cluster.conf (whether using ccs, luci, or hand-editing) will need to learn:
** The crm and/or pcs commands and become familiar with the CIB, which is an XML database of the failover resource configuration combined with some run-time state information.
** The layout, syntax, and use of corosync.conf
* Users who relied on the clustat command will need to learn the following commands:
** cibadmin
** crm_mon
** crm_resource


== Dependencies ==
== Dependencies ==


* lvm2-cluster
* corosync (2.0)
* asterisk (openais plugin)
* pacemaker
* qpidc (corosync libcpg user)
* cluster-glue
* dlm (in review currently)
* libqb
* pcs (in review currently)


All packagers and upstreams have been informed upfront and worked together to achieve this goal.
All packagers and upstreams have been informed upfront and worked together to achieve this goal.
All changes are already part of the Fedora archive as rawhide build 9 Jul 2009 (rawhide report: 20090709 changes confirms).


== Contingency Plan ==
== Contingency Plan ==
Line 61: Line 98:
== Documentation ==
== Documentation ==


* See [https://fedoraproject.org/wiki/Docs/Beats/Cluster Cluster Fedora 11 Beats]
* [http://www.clusterlabs.org/doc/en-US/Pacemaker/1.1/html/Clusters_from_Scratch/ Clusters from Scratch]
* [http://corosync.org/doku.php?id=faq Corosync FAQ]


== Release Notes ==
== Release Notes ==
In addition to the [https://fedoraproject.org/wiki/Docs/Beats/Cluster cluster features present] in Fedora 11, Fedora 12 includes the following enhancements:
* The update to version 2.0 (needle) of the [http://www.corosync.org/doku.php Corosync Cluster Engine] offers:
 
** API stability guarantees for the duration of Corosync 2.0's lifetime
* The update to version 1.0 (flatiron) of the [http://www.corosync.org/doku.php Corosync Cluster Engine] offers (on top of Fedora 11's features):
** High performance, cluster-wide messaging.
** Plugin API stability guarantees for the duration of Corosync 1.0's lifetime
*** The performance of corosync 2.0 is nearly 10 times that of corosync 1.x for smaller (<64kb) message sizes
** High performance messaging
*** The CPU utilization of corosync 2.0 is approximately 1/5th of corosync 1.x
** Ultra-high performance IPC subsystem available to other components and developers
** Improved quorum subsystem
** Rolling upgrade compatibility with 0.80.z releases of [http://www.openais.org/ OpenAIS]
** The confdb API has been replaced with the cmap API for configuration management.
* The update to version 3.0 of the [http://sources.redhat.com/cluster Linux-Cluster Project] offers:
* The update to version 1.1.7 of the Pacemaker Cluster Resource Manager offers:
** Reduced code and operational complexity compared to version 2 leading to greater reliability
** Improved fencing subsystem which can be used in either GFS2 standalone environments or resource-driven clusters (which may or may not include GFS2)
** Pluggable, simplified configuration management framework with plugins for XML and LDAP (experimental), including schemas for both classes
** Improved logging via the libqb framework
** Improved code quality due to stricter coding guidelines
* The inclusion of [http://www.clusterlabs.org pacemaker] 1.0 offers (note: packages awaiting reviews):
** Improved resource & failover management over rgmanager
*** Resource-dependency model allows for flexible placement of cluster resources
*** Can run rgmanager's resource agents with minimal changes, offering an upgrade path for users of rgmanager
** Fine-grained resource management: administrators may operate on one resource instead of operating at the resource group level


== Comments and Discussion ==
== Comments and Discussion ==
* See [[Talk:Features/Cluster]]
* See [[Talk:Features/Cluster]]


[[Category:FeaturePageIncomplete]]
[[Category:FeatureAcceptedF17]]

Revision as of 08:01, 26 March 2012

Updated Cluster Stack with Enhanced Failover Support

Summary

This is a significant update to the clustering stack - both for high availability and load balancing.

Owner

Current status

  • Targeted release: Fedora 40
  • Last updated: 26 Mar 2012
  • Percentage of completion: 100%

Detailed Description

Upstream has been developing a new version of the cluster stack which is currently shipped with Fedora 16. The major features are:

  • Improved quorum subsystem which is integrated in to the Corosync Cluster Engine (95% complete)
  • A new command-line interface for administration of both the Corosync Cluster Engine and the Pacemaker Cluster Resource Manager as well as monitoring (50% complete)
  • Convergence on a single set of resource agents from the Linux clustering community
  • Enhanced fencing support provided by stonith-ng, which is part of the Pacemaker project.
  • Separation of GFS2-specific utilities in to a separate project (100%)
  • Separation of DLM-specific utilities in to a separate project (90%)

Note that this transition includes deprecations:

  • The cluster package, including CMAN, DLM utilities, GFS2 utilities, fenced, and all associated libraries.
  • The rgmanager package. All users are advised to use Pacemaker for their failover needs.
  • The openais package. This provided some of the SA Forum AIS APIs, which were rarely used.
  • The heartbeat package. Upstream development has stopped and equivalent or superior functionality is provided by the Corosync package.
  • The piranha package. Superior functionality is provided by the Keepalived package.
  • The ricci package. Similar agent-style communication can be provided by Matahari or other projects.
  • The luci package. This will be replaced by Sunzi, although it is not clear this part will make Fedora 17.
  • The clustermon package. This was part of the ricci/luci administration stack and is no longer required.
  • Several agents in the resource-agents package which previously relied on examination of cluster.conf at run-time.

Benefit to Fedora

The increased reliability and versatility of the cluster components included in Fedora 17 allow administrators to deploy Fedora in environments where greater availability and clustered file systems are required. Removal of rgmanager matches the capabilities offered by several other Linux distributions and allows the Fedora community to consort efforts on a single failover stack.

Scope

This change impacts several utilities and other applications, including:

  • The DLM, which used AIS Checkpoints to replicate data about POSIX locks (100% complete)
  • Clustered LVM, which used CMAN APIs, will need to be ported to corosync. (?% complete)
  • Fence-virt, which used CMAN APIs and AIS Checkpoints, will need to be ported to corosync. (0% complete)
  • Pacemaker, which used CMAN APIs, will need to be ported to corosync (100% complete)
  • Clustered mirroring, which uses AIS Checkpoints will need to be ported to corosync (?% complete)
  • QPid, which used CMAN APIs, will need to be ported to corosync (?% complete)

How To Test

The initial use cases required for high availability are documented in the Clusters from Scratch manual.

Information about how to run GFS2 in standalone mode will be provided as development nears completion.

User Experience

Some differences will be noticeable by users:

  • Users of the rgmanager package will need to learn the Pacemaker package, which is a more dynamic, complex, full-featured cluster resource manager.
  • Users who relied on cluster.conf (whether using ccs, luci, or hand-editing) will need to learn:
    • The crm and/or pcs commands and become familiar with the CIB, which is an XML database of the failover resource configuration combined with some run-time state information.
    • The layout, syntax, and use of corosync.conf
  • Users who relied on the clustat command will need to learn the following commands:
    • cibadmin
    • crm_mon
    • crm_resource

Dependencies

  • corosync (2.0)
  • pacemaker
  • cluster-glue
  • dlm (in review currently)
  • libqb
  • pcs (in review currently)

All packagers and upstreams have been informed upfront and worked together to achieve this goal.

Contingency Plan

None necessary

Documentation

Release Notes

  • The update to version 2.0 (needle) of the Corosync Cluster Engine offers:
    • API stability guarantees for the duration of Corosync 2.0's lifetime
    • High performance, cluster-wide messaging.
      • The performance of corosync 2.0 is nearly 10 times that of corosync 1.x for smaller (<64kb) message sizes
      • The CPU utilization of corosync 2.0 is approximately 1/5th of corosync 1.x
    • Improved quorum subsystem
    • The confdb API has been replaced with the cmap API for configuration management.
  • The update to version 1.1.7 of the Pacemaker Cluster Resource Manager offers:
    • Improved fencing subsystem which can be used in either GFS2 standalone environments or resource-driven clusters (which may or may not include GFS2)
    • Improved logging via the libqb framework

Comments and Discussion