From Fedora Project Wiki
 
(24 intermediate revisions by 2 users not shown)
Line 16: Line 16:
== Current status ==
== Current status ==
* Targeted release: [[Releases/18 | Fedora 18 ]]  
* Targeted release: [[Releases/18 | Fedora 18 ]]  
* Last updated: 2012-07-23
* Last updated: 2012-09-17
* Percentage of completion: 30 %
* Percentage of completion: 100 %


== Detailed Description ==
== Detailed Description ==
= Publish OS information to Service Processor =
'''Fedora18'''
* '''1. Publish OS information to Service Processor - 100%'''
** '''Purpose''': OS information should be accessible via the Service Processor remotely.
** '''Purpose''': OS information should be accessible via the Service Processor remotely.
** On systems that contain a service processor, upon each boot-up, publish these to the Service Processor:
** On systems that contain a service processor, upon each boot-up, publish these to the Service Processor:
Line 26: Line 27:
*** “OS Version”, Example: “17"
*** “OS Version”, Example: “17"
*** “System Host Name”, Example: “fedora.example.com”
*** “System Host Name”, Example: “fedora.example.com”
** Achieved by a mix of standard IPMI and delloem commands
** os-name, os-hostname are standard IPMI commands. os-version is OEM command.


= Heartbeat to Service Processor =
* '''2. Heartbeat to Service Processor - 100%'''
** '''Purpose''': Capture screen shot in Service Processor for debugging on system crash during runtime and install-time
** '''Purpose''': Capture screen shot in Service Processor for debugging on system crash during runtime and install-time
** On each startup or during installation, set-up the IPMI watchdog via freeipmi-bmc-watchdog systemd service
** On each startup or during installation, set-up the IPMI watchdog via systemd's hardware watchdog. Example:  RuntimeWatchdogSec=120
** During runtime, setting IPMI_WATCHDOG=yes in /etc/sysconfig/ipmi (OpenIPMI package) should enable systemd watchdog


=Retrieve log from Service Processor=
* '''3. Support for redirection of SNMP - 100%'''
** '''Purpose''': Have syslog log Service Processor events so there is one log where system administrators can look for OS and Service Processor events.
** OS daemon should fetch logs from Service Processor, preferably with filtering capability.
** In addition to what ipmievd can provide currently
 
=Support for redirection of SNMP=
** '''Purpose''': Utilize SNMP agent on Service Processor and provide access to Service Processor MIB via the OS's SNMP agent.
** '''Purpose''': Utilize SNMP agent on Service Processor and provide access to Service Processor MIB via the OS's SNMP agent.
** Redirect selected snmp queries to the service processor's IP
** Redirect selected snmp queries to the service processor's IP
Line 44: Line 41:
** Example on Dell PowerEdge Servers: Redirect OIDs under .1.3.6.1.4.1.674.10892.2 to the service processor, we will need this in /etc/snmpd.conf: proxy -v2c -Os -c public <Serv_Proc_IP> .1.3.6.1.4.1.674.10892.2
** Example on Dell PowerEdge Servers: Redirect OIDs under .1.3.6.1.4.1.674.10892.2 to the service processor, we will need this in /etc/snmpd.conf: proxy -v2c -Os -c public <Serv_Proc_IP> .1.3.6.1.4.1.674.10892.2


=Include IPMI support in anaconda=
'''Fedora 19'''
* '''4. Include IPMI support in anaconda - Work in progress'''
** '''Purpose''': Set/Retrieve systems management information during install-time
** '''Purpose''': Set/Retrieve systems management information during install-time
** Use-case: Communicate various stages of the anaconda installation to the Service Processor to aid in debugging installation issues.
** Use-case: Communicate various stages of the anaconda installation to the Service Processor to aid in debugging installation issues.
** Use-case: Capture screen-shot from Service Processor in case of install-time system crash/hang.
** Use-case: Capture screen-shot from Service Processor in case of install-time system crash/hang.
** Development: http://www.redhat.com/archives/anaconda-devel-list/2012-July/msg00101.html


=Publish Service Processor URL and IP via WSMAN=
* '''5. Retrieve log from Service Processor '''
** '''Purpose''': Have syslog log Service Processor events so there is one log where system administrators can look for OS and Service Processor events.
** OS daemon should fetch logs from Service Processor, preferably with filtering capability.
** In addition to what ipmievd can provide currently
 
* '''6. Publish Service Processor URL and IP via WSMAN - 50%'''
** '''Purpose''': One-to-Many management consoles are able to launch the service processor management console by retrieving the URL from an OS based agent. Moving this functionality into the OS enables the same feature without the need to install an additional application into the OS.
** '''Purpose''': One-to-Many management consoles are able to launch the service processor management console by retrieving the URL from an OS based agent. Moving this functionality into the OS enables the same feature without the need to install an additional application into the OS.
** Retrieve IP address and URL of service processor and expose them via standard DMTF name-space
** Retrieve IP address and URL of service processor and expose them via
*** standard DMTF name-space
*** environment variable for privileged users
** Needs to be dynamic: Any changes to service processor IP address/URL should reflect on the host OS or when queried by wsman
** Needs to be dynamic: Any changes to service processor IP address/URL should reflect on the host OS or when queried by wsman
** Can use existing CIM name space: https://sblim.sf.net/wbem/wscim/1/cim-schema/2
** Can use existing CIM name space: https://sblim.sf.net/wbem/wscim/1/cim-schema/2
** '''Status'''
*** Completed fetch and set environment variable
*** Working upstream (openwsman) on exposing variables via WSMAN


=WS-MAN Provider Redirection=
* '''7. WS-MAN Provider Redirection'''
** '''Purpose''': Many management consoles and tools manage hardware via WS-MAN.  This requires the addition of a WMI provider from the hardware vendor.  Placing a WS-MAN redirection to the service processor’s WS-MAN stack into the OS enables the same feature without the need to install an additional application into the OS
** '''Purpose''': Many management consoles and tools manage hardware via WS-MAN.  This requires the addition of a WMI provider from the hardware vendor.  Placing a WS-MAN redirection to the service processor’s WS-MAN stack into the OS enables the same feature without the need to install an additional application into the OS
** Development: http://sourceforge.net/mailarchive/message.php?msg_id=29640731


== Benefit to Fedora ==
== Benefit to Fedora ==
Line 63: Line 73:


== Scope ==
== Scope ==
The new features will require additions/modifications to:
The new features will require:
* Fedora’s start-up scripts and default configuration files of certain services
* Automated loading of ipmi driver where service processor hardware is available.
* Support for IPMI in Anaconda
** ipmi_si
** ipmi_devintf
** ipmi_msghandler
** Proposed: https://lkml.org/lkml/2012/7/26/278
** Contingency: OpenIPMI already has systemd start-up script that loads the drivers when enabled.
* One start-up script that will run after ipmi drivers have loaded to:
** fetch service processor IP address/URL
** set OS name, version in the service processor
** setup snmpd.conf for redirection
* A configuration file that accompanies the start-up script that contains:
** snmpd OID of the service processor (will differ for each OEM)
* systemd already has support for hardware watchdog, but we will require that ipmi_watchdog driver is loaded and does not conflict with iTCO_wdt on systems that have both watchdog hardware.
** This is not Dell specific and will work on any system with an IPMI compliant Service Processor
** Patch /usr/libexec/openipmi-helper to set systemd watchdog if /etc/sysconfig/ipmi:IPMI_WATCHDOG=yes
* Support for IPMI (driver and freeipmi) in Anaconda
** systemd already supports hardware watchdog.


== How To Test ==
== How To Test ==
Line 88: Line 113:
== Dependencies ==
== Dependencies ==
* systemd
* systemd
* freeipmi
* OpenIPMI
* freeipmi-bmc-watchdog
* ipmitool
* ipmitool
* anaconda
* anaconda
* dracut
* dracut
* openwsman


== Contingency Plan ==
== Contingency Plan ==
Line 103: Line 128:


== Release Notes ==
== Release Notes ==
* Agent-free systems management of Fedora system remotely via Service Processor.
* Better management of Fedora system remotely via Service Processor
** On systems that contain IPMI compliant Service Processors, it is now possible to have closer integration of OS and Service Processor without the need for 3rd party software. This will enable better management of the system remotely.


== Comments and Discussion ==
== Comments and Discussion ==
* See [[Talk:Features/AgentFreeManagement]]
* See [[Talk:Features/AgentFreeManagement]]
* [https://lists.us.dell.com/mailman/listinfo/linux-poweredge/ linux-poweredge] mailing list.
* [https://lists.us.dell.com/mailman/listinfo/linux-poweredge/ linux-poweredge] mailing list.
[[Category:FeatureAcceptedF18]]
<!-- When your feature page is completed and ready for review -->
<!-- remove Category:FeaturePageIncomplete and change it to Category:FeatureReadyForWrangler -->
<!-- After review, the feature wrangler will move your page to Category:FeatureReadyForFesco... if it still needs more work it will move back to Category:FeaturePageIncomplete-->
<!-- A pretty picture of the page category usage is at: https://fedoraproject.org/wiki/Features/Policy/Process -->

Latest revision as of 10:06, 17 September 2012

Agent-Free Systems Management

Summary

On server class systems, monitoring and managing hardware health/configuration remotely for large number of systems is crucial. One important component of this systems management solution on each server is the Service Processor. System Administrators and Monitoring/Configuration Software (like Nagios, etc.) connect to the Service Processor via shared/dedicated management networks.

The information provided by the Service Processor is mostly independent of the Operating System running on the server. It is possible through systems management software installed on the operating system to obtain a richer set of systems management functionality overall. Such systems management software that run on Linux are specific to the vendor of the server, and can also be proprietary. They can also be bulky and require to be validated/managed like any other application.

We can envision an ideal systems management solution comprising of the Service Processor and the operating system combination that “just work” without the need for a vendor specific (and sometimes proprietary) software without a major loss of feature.

The goal of this feature is the substitute some of the important functionality of the systems management software that is usually installed on the operating system by a native implementation. This will also put existing standards already in use by Service Processors like IPMI and WSMAN to better use.

Owner

Current status

  • Targeted release: Fedora 18
  • Last updated: 2012-09-17
  • Percentage of completion: 100 %

Detailed Description

Fedora18

  • 1. Publish OS information to Service Processor - 100%
    • Purpose: OS information should be accessible via the Service Processor remotely.
    • On systems that contain a service processor, upon each boot-up, publish these to the Service Processor:
      • “OS Name”, Example: “Fedora”
      • “OS Version”, Example: “17"
      • “System Host Name”, Example: “fedora.example.com”
    • os-name, os-hostname are standard IPMI commands. os-version is OEM command.
  • 2. Heartbeat to Service Processor - 100%
    • Purpose: Capture screen shot in Service Processor for debugging on system crash during runtime and install-time
    • On each startup or during installation, set-up the IPMI watchdog via systemd's hardware watchdog. Example: RuntimeWatchdogSec=120
    • During runtime, setting IPMI_WATCHDOG=yes in /etc/sysconfig/ipmi (OpenIPMI package) should enable systemd watchdog
  • 3. Support for redirection of SNMP - 100%
    • Purpose: Utilize SNMP agent on Service Processor and provide access to Service Processor MIB via the OS's SNMP agent.
    • Redirect selected snmp queries to the service processor's IP
    • Redirect traps form Service Processor to Fedora's trap destination.
    • Detect Service Processor interface IP and configure SNMP redirection and trap configuration
    • Example on Dell PowerEdge Servers: Redirect OIDs under .1.3.6.1.4.1.674.10892.2 to the service processor, we will need this in /etc/snmpd.conf: proxy -v2c -Os -c public <Serv_Proc_IP> .1.3.6.1.4.1.674.10892.2

Fedora 19

  • 4. Include IPMI support in anaconda - Work in progress
    • Purpose: Set/Retrieve systems management information during install-time
    • Use-case: Communicate various stages of the anaconda installation to the Service Processor to aid in debugging installation issues.
    • Use-case: Capture screen-shot from Service Processor in case of install-time system crash/hang.
    • Development: http://www.redhat.com/archives/anaconda-devel-list/2012-July/msg00101.html
  • 5. Retrieve log from Service Processor
    • Purpose: Have syslog log Service Processor events so there is one log where system administrators can look for OS and Service Processor events.
    • OS daemon should fetch logs from Service Processor, preferably with filtering capability.
    • In addition to what ipmievd can provide currently
  • 6. Publish Service Processor URL and IP via WSMAN - 50%
    • Purpose: One-to-Many management consoles are able to launch the service processor management console by retrieving the URL from an OS based agent. Moving this functionality into the OS enables the same feature without the need to install an additional application into the OS.
    • Retrieve IP address and URL of service processor and expose them via
      • standard DMTF name-space
      • environment variable for privileged users
    • Needs to be dynamic: Any changes to service processor IP address/URL should reflect on the host OS or when queried by wsman
    • Can use existing CIM name space: https://sblim.sf.net/wbem/wscim/1/cim-schema/2
    • Status
      • Completed fetch and set environment variable
      • Working upstream (openwsman) on exposing variables via WSMAN
  • 7. WS-MAN Provider Redirection
    • Purpose: Many management consoles and tools manage hardware via WS-MAN. This requires the addition of a WMI provider from the hardware vendor. Placing a WS-MAN redirection to the service processor’s WS-MAN stack into the OS enables the same feature without the need to install an additional application into the OS
    • Development: http://sourceforge.net/mailarchive/message.php?msg_id=29640731

Benefit to Fedora

  • The Fedora users of servers that contain Service Processors do not have to install additional software dedicated to systems management and still expect standard pieces of information to be available remotely.
  • Assists with debugging system failures (panic, hang, etc.) remotely.

Scope

The new features will require:

  • Automated loading of ipmi driver where service processor hardware is available.
    • ipmi_si
    • ipmi_devintf
    • ipmi_msghandler
    • Proposed: https://lkml.org/lkml/2012/7/26/278
    • Contingency: OpenIPMI already has systemd start-up script that loads the drivers when enabled.
  • One start-up script that will run after ipmi drivers have loaded to:
    • fetch service processor IP address/URL
    • set OS name, version in the service processor
    • setup snmpd.conf for redirection
  • A configuration file that accompanies the start-up script that contains:
    • snmpd OID of the service processor (will differ for each OEM)
  • systemd already has support for hardware watchdog, but we will require that ipmi_watchdog driver is loaded and does not conflict with iTCO_wdt on systems that have both watchdog hardware.
    • This is not Dell specific and will work on any system with an IPMI compliant Service Processor
    • Patch /usr/libexec/openipmi-helper to set systemd watchdog if /etc/sysconfig/ipmi:IPMI_WATCHDOG=yes
  • Support for IPMI (driver and freeipmi) in Anaconda
    • systemd already supports hardware watchdog.

How To Test

  • Install Fedora on test machine with service processor
  • Publish OS information to Service Processor
    • Service Processor should provide the OS Version and Name via various supported interfaces
  • Heartbeat to Service Processor
    • With the watchdog daemon configured, a kernel panic or system crash should result in the system rebooting after the set time and a snapshot of the crash and/or an entry in the SEL log should be recorded.
  • Retrieve log from Service Processor
    • syslog should contain IPMI SEL events logged by ipmievd
  • Support for redirection of SNMP
    • After configuration of /etc/snmpd.conf, snmp queries to Fedora with the Service Processor OID should succeed and return correct values that would otherwise be retrieved via the Service Processor's snmp agent.
  • Include IPMI support in anaconda
    • During install time, we should have access to ipmitool or freeipmi commands that can be used via kickstart's pre-install section.
  • Publish Service Processor URL and IP via WSMAN
    • Get Service Processor URL and IP by querying the OS's wsman server.

User Experience

  • Easier management of larger networks using industry standard technologies.
  • Minimal intervention from user in configuring Service Processor access details in the OS.

Dependencies

  • systemd
  • OpenIPMI
  • ipmitool
  • anaconda
  • dracut
  • openwsman

Contingency Plan

  • Document with script-lets on how similar functionality can be achieved.

Documentation

Release Notes

  • Better management of Fedora system remotely via Service Processor
    • On systems that contain IPMI compliant Service Processors, it is now possible to have closer integration of OS and Service Processor without the need for 3rd party software. This will enable better management of the system remotely.

Comments and Discussion