This is an evaluation by the Big Data SIG of the issues that need to be addressed in order to get Ambari into Fedora. This work was started on the 1.2.5 branch, which didn't support Hadoop 2.x at the time. The current stable release (1.4.4) does.
Issues To Be Resolved
Missing node.js Dependencies
The Ambari build uses brunch and other node.js parts to generate static web content. A significant part of the development dependency chain for the node.js parts for brunch and ember are not in Fedora and 53 node.js projects would need to be packaged. There are a few ways to handle the node.js dependency chain:
- Package all the node.js dependencies as individual rpms
- Package all the node.js dependencies as a single rpm
- Work with upstream to remove the need to generate the static web content
- Re-implement the node.js parts in source native to Ambari
- Find similar functionality that is already packaged in Fedora and provide support for its use in the Ambari build
- Abandon packaging Ambari for Fedora
Missing Java Dependencies
There are only 2 Java dependencies that aren't in Fedora that Ambari needs, both for the test phase.
This is a module of powermock which is already packaged in Fedora, however the powermock package currently disables the easymock module. A BZ for this issue has been raised since the latest version of easymock appears to be compatible.
Dependency Version Issues
Ambari uses puppet manifests and directives for provisioning Hadoop components on hosts. At build, puppet version 2.7.9 is downloaded from Puppet Labs and added to the agent package. However, the version that is currently available in rawhide is 3.4.3. The puppet parser validates a config at application so this poses problems in two areas (so far):
- Agent modules have variable declarations like "$core-site=...". Version 3.4.3 forbids hyphens in variable names (alphanumeric and underscore only).
- At install, the agent retrieves puppet manifests for the selected stack (e.g., HDP 2.0.6). The structure of those cannot be processed by version 3.4.3 and it fails validation with "Import loop detected" errors.
The last version of puppet 2.7 built for Fedora still builds in rawhide at this point in time. However, it obviously would have to replace the incumbent version (3.4.x) due to files in common.
Facter and Ruby
Rawhide currently has Facter 1.7.4 and Ruby 2.0.0 (deps for Puppet) while the Ambari build bundles older versions of both in the agent. Ruby 2.0.0 is not compatible with the older version of Puppet.
The current version of Jetty in Fedora is Jetty 9, but Ambari is coded for Jetty 7. Fedora now has a Jetty 8 compatibility package in rawhide and necessary modifications are here.
The version of postgres in Fedora may require updates to the database initialization done by Ambari. There is an upstream patch to address this, which appears to have been fixed for 1.5.0 (yet to be released).
The runtime is actually designed and implemented to search for locally, and (if necessary) download the Oracle JDK 6 from the Hortonworks site (specifically jdk-6u31-linux-x64.bin). A command-line argument can be passed to the ambari-server setup task (-j <jvm location>) that will use that JVM path uniformly for both agents and the server. Thus, OpenJDK7 is technically supported though not the default.
Hadoop 2.x Support
The current Ambari release (1.4.4) supports the Hadoop 2.0.6 HDP release; Fedora has 2.2.0. Due to the nature of executing a downloaded HDP stack from Hortonworks, it is unknown at this time if there are specific compatibility issues with 2.2.0.
The Ambari build uses the rpm-maven-plugin to generate rpms. Obviously, this maven plugin doesn't exist in Fedora and likely never will since it is antithetical to Fedora packaging from specifications. A Fedora spec build can ignore the presence of this plugin and just use artifacts as they sit in BUILD, but it does represent a significant disconnect between upstream and Fedora.
Fedora Packaging Repository
Ambari has the ability to install packages on a client machine and it pulls those packages from Hortonworks repos that are hard coded in the server. It determines which repositories to use based upon the OS, and Fedora is not recognized as a valid/supported OS. Ambari will need to be modified to not only accept Fedora as a valid OS, but also to pull the packages from Fedora repos and not from Hortonworks. This Fedora specific issue has been logged with upstream, as has a more general architecture request for CDH and Apache repos.