Packaging:AutoProvidesAndRequiresFiltering

From FedoraProject

Revision as of 17:31, 14 April 2011 by Tibbs (Talk | contribs)

Jump to: navigation, search

Contents

Summary

RPM has no general or standard mechanism to enable filtering of auto-generated requires and provides; this guideline describes how Fedora has implemented such a system.

  • MUST: Packages must not provide RPM dependency information when that information is not global in nature, or are otherwise handled (e.g. through a virtual provides system). e.g. a plugin package containing a binary shared library must not "provide" that library unless it is accessible through the system library paths.
  • MUST: When filtering automatically generated RPM dependency information, the filtering system implemented by Fedora must be used, except where there is a compelling reason to deviate from it.
  • MUST NOT: If the package does not fall into one of the categories specified in the Usage Section below then it must not filter the provides at this time. The reason for this is explained in the Usage Section.

Rationale

The auto requires and provides system contained in RPM is quite useful; however, it often picks up "private" package capabilities that shouldn't be advertised as global, things that are "just wrong", or things prohibited by policy (e.g. deps from inside %{_docdir}).

For example:

  • Various "plugin" packages (e.g. Pidgin, Perl, Apache, KDE) are marked as "providing" private shared libraries outside the system path.
  • Files in %{_docdir} are routinely scanned, and can trigger prov/req when this is explicitly forbidden by policy.

As it stands, filtering these auto-generated requires and provides is difficult and messy at best, and horribly deep magic in many cases; with little guidance on how to do it. This feature aims to make the following tasks easy:

  • preventing files/directories from being scanned for requires (pre-scan filtering)
  • preventing files/directories from being scanned for provides (pre-scan filtering)
  • removing items from the requires stream (post-scan filtering)
  • removing items from the provides stream (post-scan filtering)

There are three different filtering mechanisms available:

  • The recommended mechanism, available beginning with rpm 4.9.0 (Fedora 15 and newer).
  • A version with some drawbacks, needed for Fedora 13, Fedora 14 and EPEL 6.
  • An old method needed for EPEL 4 and 5, which is not covered by this document. See EPEL:Packaging#Perl_Provides_and_Requires_Filtering.

RPM 4.9 method

Usage

These filtering macros can be used with any package.

Location of macro invocation

It's strongly recommended that these filtering macros be invoked before %description, but after any other definitions. This will keep them in a consistent place across packages, and help prevent them from being mixed up with other sections.

Printing files/directories from being scanned for provides (pre-scan filtering)

The %__provides_exclude_from macro is used to specify a regular expression matching files or directories that should not be scanned for any "provides" information. You can only define it once, so if you need to match multiple locations you will need to construct a compound pattern. Note that the buildroot (i.e. %{buildroot} or $RPM_BUILD_ROOT') is stripped from the pathname before comparison.

XXX What regexp language is in use? XXX Is there any means of or need to escape characters from macros before using them in these patterns? XXX Is "macro" the proper term for these things? XXX Include example of a compound pattern. Depends on the regexp language in use.

We can filter by regex:

%define __provides_exclude_from %{perl_vendorarch}/.*\.so$

Or by anything matching, say, a directory:

%define __provides_exclude_from %{_docdir}

==== Preventing files/directories from being scanned for requires (pre-scan filtering) The %__requires_exclude_from macro is used to specify a regular expression matching files or directories that should not be scanned for any "requires" information; it does for requires what the %__provides_exclude_from macro does for provides and is defined in the same fashion.

Removing items from the provides stream (post-scan filtering)

Post-scan provides filtering is specified with the '%__provides_exclude macro. Simply set this macro to a regular expression and all matching "provides" will be removed.

For example, if we're finding that the auto-prov system is finding an incorrect provide, we can filter it:

%define __provides_exclude bad-provide

Since this macro can only be defined a single time, if multiple provides must be filtered, you must construct a pattern which matches them all:

%define __provides_exclude (bad-provide|another-erroneous-provide)


Removing items from the requires stream (post-scan filtering)

The %__requires_exclude macro is used to filter "requires"; it does for requires what the %__provides_exclude macro does for provides and is invoked in the same fashion.

Fedora 13, 14 and EPEL6 method

Usage

These filtering macros MUST only be used with packages which meet one of the following criteria:

  • Noarch packages
  • Architecture specific packages with no binaries in $PATH (e.g. /bin, /usr/bin, /sbin, /sbin) or libexecdir and no system libs in libdir. This includes all of the subpackages generated from the spec file.

They are not permitted in any other cases, because the macros interfere with the "coloring" of elf32/64 executables done internally by RPM to support multilib installs.

Location of macro invocation

It's strongly recommended that these filtering macros be invoked before %description, but after any other definitions. This will keep them in a consistent place across packages, and help prevent them from being mixed up with other sections.

Preventing files/directories from being scanned for provides (pre-scan filtering)

The %filter_provides_in macro is used to define the files or directories that should not be scanned for any "provides" information. This macro may be safely invoked multiple times, and can handle regular expressions. The -P flag can be passed to specify that a PCRE is being used.

We can filter by regex:

%filter_provides_in %{perl_vendorarch}/.*\.so$ 
%filter_provides_in -P %{perl_archlib}/(?!CORE/libperl).*\.so$ 

Or by anything matching, say, a directory:

%filter_provides_in %{_docdir}

Preventing files/directories from being scanned for requires (pre-scan filtering)

The %filter_requires_in macro is used to define the files or directories that should not be scanned for any "requires" information; it does for requires what the %filter_provides_in macro does for provides and is invoked in the same fashion.

Removing items from the provides stream (post-scan filtering)

Post-scan provides filtering is invoked through the %filter_from_provides. This macro can be fed a sed expression to filter from the stream of auto-found provides.

For example, if we're finding that the auto-prov system is finding an incorrect provide, we can filter it:

%filter_from_provides /bad-provide/d

Note that we should always specify this in terms of a regexp.

Removing items from the requires stream (post-scan filtering)

The %filter_from_requires macro is used to filter "requires"; it does for requires what the %filter_from_provides macro does for provides and is invoked in the same fashion.

General filter setup

The %filter_setup macro must be invoked after defining any specific overrides; this macro does all the heavy lifting of implementing the filtering desired:

# ... filtering defines here
%filter_setup

These macros were not defined in EPEL5. People wanting to share one spec file with Fedora and EPEL need to conditionalize use of the macros. That can be done like this:

%{?filter_setup:
%filter_provides_in %{python_sitearch}.*\.so$
%filter_setup
}

Simplified macros for common cases

In some cases, the filtering of extraneous Provides: is fairly generic to all packages which provide similar things. There are simple macros that setup filters correctly for those cases so that you can do the filtering with one line. If you need to filter a bit more than the simple macro provides, you still have the option to use the macros listed above.

Perl

Perl extension modules can be filtered using this macro:

%{?perl_default_filter}

This is equivalent to:

%filter_provides_in %{perl_vendorarch}/.*\\.so$ 
%filter_provides_in -P %{perl_archlib}/(?!CORE/libperl).*\\.so$ 
%filter_from_provides /perl(UNIVERSAL)/d; /perl(DB)/d 
%filter_provides_in %{_docdir} 
%filter_requires_in %{_docdir} 
%filter_setup 

Examples

Pidgin plugin package

On a x86_64 machine, the pidgin-libnotify provides pidgin-libnotify.so()(64bit), which it shouldn't, as this library is not inside the paths searched by the system for libraries; that is, it's a private, not global, "provides" and as such must not be exposed globally by RPM.

To filter this out, we could use:

%{?filter_setup:
%filter_provides_in %{_libdir}/purple-2/.*\.so$
%filter_setup
}

Arch-specific extensions to scripting languages

e.g. to ensure an arch-specific perl-* package won't provide or require things that it shouldn't, we could use an invocation as such:

# we don't want to provide private Perl extension libs
%{?perl_default_filter}

A recipe for python:

# we don't want to provide private python extension libs
%{?filter_setup:
%filter_provides_in %{python_sitearch}/.*\.so$ 
%filter_setup
}

 %_docdir filtering

By policy, nothing under %_docdir is allowed to either "provide" or "require" anything. We can prevent this from happening by preventing anything under %_docdir from being scanned:

# we don't want to either provide or require anything from _docdir, per policy
%{?filter_setup:
%filter_provides_in %{_docdir} 
%filter_requires_in %{_docdir}
%filter_setup
}