From Fedora Project Wiki

< Changes

Revision as of 14:15, 1 July 2020 by Iucar (talk | contribs) (Submission)

FlexiBLAS as BLAS/LAPACK manager

Summary

BLAS/LAPACK packages will be compiled against the FlexiBLAS wrapper library, which will set OpenBLAS as system-wide default backend, and at the same time will provide a proper switching mechanism that currently Fedora lacks.

Owner

Current status

  • Targeted release: Fedora 33
  • Last updated: 2020-07-01
  • FESCo issue: <will be assigned by the Wrangler>
  • Tracker bug: <will be assigned by the Wrangler>
  • Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

BLAS and LAPACK are API standards for basic linear algebra operations (such as vector and matrix multiplication). Fedora packages the reference implementation from Netlib as well as several optimized backends, such as ATLAS, BLIS and OpenBLAS. Historically, there have been two unresolved issues regarding this API:

  1. Fedora lacks a system-wide default. Due to implementation differences, it is important that all components of a particular software stack link to the same BLAS/LAPACK implementation. Although there have been efforts to standardize OpenBLAS as a system-wide default, they are incomplete.
  2. Fedora lacks a proper switching mechanism. Different implementations work differently depending on the workload, but also depending on whether the consumer of this API implements some kind of parallelization, for instance. Therefore, users may want to choose a particular implementation that works best for them at run time. Mechanisms such as update-alternatives and modules have been discussed in the past, but were considered improper (the former) or faced technical issues (the former).

This change proposal aims to solve both issues at the same time with minimal changes and minimal impact by using FlexiBLAS.

FlexiBLAS is a framework that wraps the BLAS/LAPACK API with interfaces for both 32- and 64-bit integers. It provides runtime exchangeable backends without recompilation, with a transparent fallback mechanism to Netlib's reference implementation if a certain symbol is not present in the selected backend. It also supports flexible per-system/user/host configuration files and basic profiling.

Therefore, this change proposal requires:

  1. Recompilation of all BLAS/LAPACK-dependent packages linking against FlexiBLAS instead of the current implementation they are using (just changing a BuildRequires line should be sufficient in most cases, unless a SPEC has something hardcoded somewhere else).
  2. Changing the packaging guidelines to reflect this new requirement for BLAS/LAPACK consumers (there is a draft already.

In this way, all the BLAS/LAPACK-dependent packages will automatically use the system-wide default backend, set and centralized in the FlexiBLAS SPEC, and will benefit from the straightforward switching mechanism that FlexiBLAS provides.

Feedback

TBD

Benefit to Fedora

  • Packaging of BLAS/LAPACK-dependent packages will be easier and safer. FlexiBLAS is 100% compatible with the reference API. It uses optimized symbols that are present in the selected backend and transparently falls back to Netlib for the missing ones, so compatibility is always guaranteed.
  • Provides a centralized way of managing the system-wide default, just by changing a global variable in the FlexiBLAS SPEC.
  • Provides sysadmins with tools to manage the system-wide configuration. FlexiBLAS ships system-wide configuration files to, e.g., change the default or point to new backends. A CLI tool is also provided to easily manage such configurations.
  • Provides users with tools to switch the backend. This can be managed by the same CLI tool too in user mode, and it generates per-user configuration files.
  • Provides developers with an API to hook into the BLAS/LAPACK API calls. E.g., FlexiBLAS comes with a plugin that enables basic profiling support.

Scope

  • Proposal owners: Modify the SPECs of the BLAS/LAPACK-dependent packages to build against FlexiBLAS instead of the current backend they are using.
  • Other developers: Maintainers of the affected packages need to merge the changes and rebuild. Alternatively, a provenpackager could help with this process (it's 30-40 packages).
  • Release engineering: N/A (not needed for this Change)
  • Policies and guidelines: #995 Package guidelines should be updated to require BLAS/LAPACK-dependent packages to link against FlexiBLAS.
  • Trademark approval: N/A (not needed for this Change)

Upgrade/compatibility impact

There is no upgrade/compatibility impact.

How To Test

BLAS/LAPACK-dependent packages should work normally, without any (new) issue. To test the switching capabilities, testers may want to run a BLAS/LAPACK benchmark. There are many out there for different software, but we will describe the workflow for R as an example:

  • Install R, which requires BLAS/LAPACK for operations with matrices.
  • Run any benchmark involving calculations with matrices (e.g., R-benchmark-25.R provided by an R-core member here).
  • Switch to the reference implementation:
flexiblas -u default netlib
  • Run the benchmark again. Timings should be much worse.

User Experience

Users will have a new CLI tool, called flexiblas, which will allow them to properly switch the BLAS/LAPACK backend without administrative privileges and any compatibility issues.

Dependencies

Packages depedent on the BLAS/LAPACK API will depend on FlexiBLAS after this change. Maintainers of those packages are required to collaborate to change their BuildRequires or, alternatively, this can be done with the help of a provenpackager.

Contingency Plan

  • Contingency mechanism: Revert BLAS/LAPACK-dependent packages to their prior BuildRequires and rebuild them.
  • Contingency deadline: Beta freeze.
  • Blocks release? No.
  • Blocks product? No.

Documentation

See the README of the upstream project and their homepage for further information about FlexiBLAS.

Release Notes

TBD

Retrieved from "https://fedoraproject.org/w/index.php?title=Changes/FlexiBLAS_as_BLAS/LAPACK_manager&oldid=580985"