SIGs/bigdata/packaging/Oozie

From FedoraProject

< SIGs | bigdata | packaging(Difference between revisions)
Jump to: navigation, search
m (Besser82 moved page Oozie packaging to SIGs/bigdata/packaging/Oozie: moved to common namespace)
Line 3: Line 3:
 
= Issues To Be Resolved =
 
= Issues To Be Resolved =
 
== Missing Java Dependencies ==
 
== Missing Java Dependencies ==
Oozie depends on a number of other parts of the Hadoop ecosystem which are not in Fedora:
+
{| class="wikitable"
# HBase
+
|+ <div id="deps">Missing/Questionable Dependencies</div>
# Hive
+
! Project !! State !! Review BZ !! Packager !! Notes
# Pig
+
|-
# Sqoop
+
| hbase
 
+
| '''<span style="color:orange">Review</span>'''
It additionally depends upon a number of Apache Hadoop jars which are not packaged in the hadoop rpm in F20 as of hadoop-2.0.5-8:
+
| {{bz|1045556}}
# hadoop-client
+
| [[User:rrati|rrati]]
# hadoop-minicluster (Needed for tests)
+
|
 +
|-
 +
| hive
 +
| '''<span style="color:blue">Active</span>'''
 +
|
 +
| [[User:pmackinn|pmackinn]]
 +
|
 +
|-
 +
| pig
 +
| '''<span style="color:darkviolet">Available</span>'''
 +
|
 +
|
 +
|
 +
|-
 +
| sqoop
 +
| '''<span style="color:darkviolet">Available</span>'''
 +
|
 +
|
 +
|
 +
|}
  
 
== Oozie Build System Issues ==
 
== Oozie Build System Issues ==

Revision as of 18:07, 10 January 2014

This is a preliminary evaluation by the Big Data SIG of the work required to get Apache Oozie into Fedora. This work has been done based upon the 3.3 release building against the hadoop 2.0.5-alpha package in Fedora 20. There are likely more issues not discussed here that won't be discovered until more of the missing dependencies are packaged for Fedora.

Contents

Issues To Be Resolved

Missing Java Dependencies

Missing/Questionable Dependencies

Project State Review BZ Packager Notes
hbase Review RHBZ #1045556 rrati
hive Active pmackinn
pig Available
sqoop Available

Oozie Build System Issues

The Ooize build system is pretty rigid and does not seem configurable. For example the build seems to always want to build bits for hadoop 1.1.1 and hadoop 2.x even if a specific hadoop version is provided at build time. Similarly it does not seem possible to disable test compilation from command line options. It appears editing of the poms will be required to disable specific modules.

The hadoop versions are also hard coded in the pom files, but this shouldn't be an issue since Fedora's java build tools ignore version requests except in the case of compatibility packages.

Dependency Version Mismatches

Jetty

Oozie depends on jetty 6.1.14. Oozie will need to be updated to support the current version of jetty in Fedora.

Tomcat

Oozie depends on tomcat 6. Oozie will need to be updated to support the current version of tomcat in Fedora.

Webapp Configuration

Similar to Hadoop's HTTPFS, oozie downloads a copy of tomcat and uses that to create a specially configured tomcat configuration for oozie. This is will likely require the tomcat shell scripts. Is so then the webapp will not be able to be packaged in Fedora until the tomcat shell scripts are packaged.