From Fedora Project Wiki

< Changes

Revision as of 09:45, 13 November 2019 by Bcotton (talk | contribs) (Current status)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Modules In Non-Modular Buildroot

Enable module default streams in the buildroot repository for modular and non-modular RPMs.

Summary

This Change (colloquially referred to as "Ursa Prime") enables the Koji build-system to include the RPM artifacts provided by module default streams in the buildroot when building non-modular (or "traditional") RPMs.

Owner

Current status

  • Targeted release: Fedora 32
  • Last updated: 2019-11-13
  • Tracker bug: #1771932
  • Release notes tracker: (not required)

Detailed Description

As a major part of the Modularity design, we have a concept of default module streams. These streams are built as modules, but the RPM artifacts they deliver are intended to be used just like non-modular RPMS. The aspirational goal is that a user of the system who never executes a module-specific command (such as dnf module install nodejs:8) should experience no meaningful changes in behavior from how they would interact with a completely non-modular system. In practice, this may mean that the informational output of package managers may indicate that modules are being enabled and used, but a user that does not have a specific reason to interact with the selection of a module stream should have that managed on their behalf.

Similarly, the experience for package maintainers of non-modular packages should be unaffected by an RPM dependency moving from the non-modular repository into a default module stream. Up to the present, this has not been the case; no module stream content has been available in the non-modular buildroot for other packages to consume. Koji builds of non-modular RPMs have had only the other non-modular RPMs from that release available to their buildroots. In contrast, building on local systems has access to both the non-modular RPMs and the RPMs from any of the default module streams. With this Change, Koji builds will have the same behavior and be able to depend on content provided by default module streams. It also enables the same behavior for Modular builds: the platform stream will now include the contents of the default module streams for each release and do not need to be explicitly specified in the modulemd buildrequires.

Note: This Change does not address the other major Modularity issue we are facing around distribution upgrades with differing default streams. When discussing this Change, please keep that topic separate.

Benefit to Fedora

This will simplify the lives of package maintainers in Fedora in two primary ways. I'll use a hypothetical example of the Node.js interpreter and a JSApp package which is capable of running on Node.js 10 or 12 (but requires newer features than are provided by Node.js 8). Additionally, the JSApp package requires the same versions of Node.js at build-time.

  • Fedora 29 ships nodejs:8, nodejs:10 and nodejs:12 module streams. The nodejs:10 stream is set as the default stream.
  • Fedora 30 ships nodejs:8, nodejs:10 and nodejs:12 module streams. The nodejs:10 stream is set as the default stream.
  • Fedora 31 ships nodejs:10 and nodejs:12 module streams. The nodejs:12 stream is set as the default stream. The nodejs:14 stream will likely become available during the F31 lifetime.
  • Fedora 32 ships nodejs:10 and nodejs:12 module streams. The nodejs:12 stream is set as the default stream. The nodejs:14 stream will likely become available during the F32 lifetime.

On Fedora 29 through 31, the Node.js package maintainer needs to build the nodejs:10 package both as a module and as a non-modular RPM in the distribution so that the JSApp package can be built. With this Change, the Node.js package maintainer in Fedora 32+ will only need to build the various Node.js streams and make one of them the default stream. The packages from it will then be added to the buildroot for non-modular packages. This will also make the packaging process somewhat more efficient, as the maintainer needs only to manage the module stream and the MBS will build it for all configured platforms.

Similarly, from the perspective of dependent maintainers, there will no longer be anxiety about needing to move their package to a module if one or more of their dependencies drops their non-modular version in favor of a default stream. Their builds will continue to work as they do today.

Scope

  • Proposal owners:
  1. Update Packaging Guidelines with requirements for module default streams
  2. Create a Pungi configuration to generate the buildroot from the default module streams.
  3. Include default_modules_scm_url in the platform virtual module specification
  4. Configure Koji tags for inheriting the new modular-defaults buildroot into the standard buildroot
  • Other developers:

Packagers of default module streams will be required to conform to the policy regarding visibility of stream artifacts. Any default module stream that is not in compliance by one week before Beta Freeze will cease to be a default stream.

  • Release engineering:
  1. https://pagure.io/releng/issue/8879 - Create pungi config for Rawhide/F32 ursa prime buildroot
  2. https://pagure.io/releng/issue/8880 - Include default_modules_scm_url in platform 31 virtual module
  3. https://pagure.io/releng/issue/8881 - Configure Koji tags for inheriting f32-modular-buildroot
  • Policies and guidelines:

The Modularity Packaging Guidelines will need to be updated to indicate the strict requirements on default streams.


  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

This change is on the build-system side of things and should not impact the upgrade process directly.

How To Test

  1. Build a modular stream
  2. Make that stream a default stream (or a buildroot override)
  3. Build a non-modular RPM that requires an artifact RPM from the modular stream.

User Experience

This should not change the end-user experience.

Dependencies

Nothing known that isn't listed in the scope.

Contingency Plan

  • Contingency mechanism: Disable the buildroot inheritance in Koji to revert to the current behavior.
  • Contingency deadline: Must be complete or reverted in time for the Mass Rebuild.
  • Blocks release? Ambiguous: lack of complete implementation may indirectly cause blocking issues.
  • Blocks product? No

Documentation

Release Notes

None needed, the Change is not user-facing.