Introduction
This effort aims at ensuring that Maven in Fedora can be bootstrapped, i.e. built and run with the set of current packages, at any time. We aim to preserve this feature with any change in the set of packages.
Part of the motivation for this effort lies in some of the complexities of updating some core packages required by Maven at runtime. In some cases, a change in such a core package led to breaking Maven. When that happened, neither Maven nor most of the required packages could not be rebuilt because they are built with Maven - and at this point Maven does not work.
Solving such cases required manual inspection of the problem and steps like rollback of the change, introducing a temporary change that would ensure working Maven, the required change itself, and a rollback of the temporary change. This approach would have to be discovered with each breaking modification.
This led us to creating the javapackages-bootstrap (JPB) project. It requires only very low level components like the C compiler, core utilities and OpenJDK. The project fetches the upstream core packages required for a working Maven and the build process is simplified not to use maven during its build. The result of the build of JPB is a working set of executables, libraries and configuration files required for javapackages-tools (JPT) to work. A working JPT can be then used to build Java packages in a traditional way in Fedora.
The problem of cyclic dependencies affects a specific set of packages: those that build-require themselves, either directly or transitively by having runtime Requires on other packages which in turn require the original one. Our packages, which have this property, were modified to be able to be built in two modes: non-bootstrap (default) and bootstrap mode. When bootstrap mode is set, it usually means that the package build dependencies are reduced to JPB and the build instructions are set to invoke xmvn from bootstrap rather than the package xmvn. This means we are able to avoid circular dependencies among our packages.
MBI (Maven Bootstrap Initiative) Continuous Integration
To ensure that modifying packages retains the bootstrapping property of Maven in Fedora, a specific test was implemented.
Simply put, It builds JPB, then the rest of the packages in bootstrap mode and then all the packages again in non-bootstrap mode.
When a package that implements bootstrap mode or is an Ant dependency, is changed, MBI CI ensures that Maven and Ant in Fedora will work with the modified version. The test is time-consuming and resource-intensive. The test is currently triggered manually for pull requests. There are plans to set these tests as gating for our set of packages. On top of that, the tests should be run regularly to ensure that external modifications to our buildroot, for example changes in OpenJDK do not break the bootstrap property.
When the test succeeds, it means that all the packages that were tested can be built from scratch with no pre-existing Java packages except for OpenJDK.
The tool used for testing generates test reports. These are useful in case of failed tests. In such a situation, the report should be linked as a pull request comment. The report should contain information about the first component that failed to build. This can be then used to fix the failed build.
An example can be seen on this ant pull request: the test named “MBI - Maven Bootstrap Initiative”.
Executing the test
In order to run the test yourself, you need the tool and the configuration for it
MBICI Workflow
Build the project using an appropriate JVM:
$ mvn install
To run the tool:
$ java -jar ./target/mbici-workflow.jar
This command will be aliased as mbi in the following text.
Create a suitable workspace directory and initialize it:
$ mbi init
The mbi.yaml is the main configuration file. Additional configuration files are stored as XML files under the .mbi directory.
MBICI Config
In order to execute the workflow on the current state of the Rawhide packages, overwrite mbi.yaml with the content of the YAML configuration file from the mbici-config repository.
If the test fails, relevant logs are stored inside resultDir in the particular directory with the corresponding name. This includes files created during a normal rpm build. After modifying the files, you can rerun the run command and it will continue from where the test failed previously.
After the test is finished, you may want to generate an HTML report, that can be done by:
$ mbi report
The generated report is the same that is used as the result of CI when a pull request is tested.
