From Fedora Project Wiki
Line 72: Line 72:


* Other developers: <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Other developers: <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
<!-- What work do other developers have to accomplish to complete the feature in time for release?  Is it a large change affecting many parts of the distribution or is it a very isolated change? What are those changes?-->
** The implementation work is isolated only to the mentioned two DNF stack components
** There is no other development work related outside of the DNF stack


* Release engineering: [https://pagure.io/releng/issues #Releng issue number] <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Release engineering: N/A (not needed for this Change)
<!-- Does this feature require coordination with release engineering (e.g. changes to installer image generation or update package delivery)?  Is a mass rebuild required?  include a link to the releng issue.
The issue is required to be filed prior to feature submission, to ensure that someone is on board to do any process development work and testing and that all changes make it into the pipeline; a bullet point in a change is not sufficient communication -->


* Policies and guidelines: N/A (not needed for this Change) <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
* Policies and guidelines:
<!-- Do the packaging guidelines or other documents need to be updated for this feature?  If so, does it need to happen before or after the implementation is done?  If a FPC ticket exists, add a link here. Please submit a pull request with the proposed changes before submitting your Change proposal. -->
** Package maintainers need to comply with the Fedora packaging guidelines related to file dependencies specification (see [https://docs.fedoraproject.org/en-US/packaging-guidelines/#_file_and_directory_dependencies here])


* Trademark approval: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
<!-- If your Change may require trademark approval (for example, if it is a new Spin), file a ticket ( https://pagure.io/Fedora-Council/tickets/issues ) requesting trademark approval from the Fedora Council. This approval will be done via the Council's consensus-based process. -->
<!-- If your Change may require trademark approval (for example, if it is a new Spin), file a ticket ( https://pagure.io/Fedora-Council/tickets/issues ) requesting trademark approval from the Fedora Council. This approval will be done via the Council's consensus-based process. -->


* Alignment with Community Initiatives:  
* Alignment with Community Initiatives: N/A (no currently active initiatives)
<!-- Does your proposal align with the current Fedora Community Initiatives: https://docs.fedoraproject.org/en-US/project/initiatives/ ? It's okay if it doesn't, but it's something to consider -->
<!-- Does your proposal align with the current Fedora Community Initiatives: https://docs.fedoraproject.org/en-US/project/initiatives/ ? It's okay if it doesn't, but it's something to consider -->


Revision as of 11:39, 23 October 2023

Important.png
Comments and Explanations
The page source contains comments providing guidance to fill out each section. They are invisible when viewing this page. To read it, choose the "view source" link.
Copy the source to a new page before making changes! DO NOT EDIT THIS TEMPLATE FOR YOUR CHANGE PROPOSAL.
Idea.png
Guidance
For details on how to fill out this form, see the documentation.
Idea.png
Report issues
To report an issue with this template, file an issue in the pgm_docs repo.


DNF: Do not download filelists by default

Important.png
This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Summary

Change the DNF behavior to not download filelists by default. These metadata, which describe all the files contained within each package, are unnecessary in the majority of use cases. Additionally, these metadata files can be large in size, leading to a significant slowdown in the user experience.

Owner

Current status

  • Targeted release: Fedora Linux 40
  • Last updated: 2023-10-23
  • [<will be assigned by the Wrangler> devel thread]
  • FESCo issue: <will be assigned by the Wrangler>
  • Tracker bug: <will be assigned by the Wrangler>
  • Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

Until now, filelists were always downloaded together with other metadata. This was hardcoded and unable to change from the outside of DNF.

With these changes, we are proposing to not download the filelists metadata by default. This can be changed through the new DNF configuration option filelists which is False by default. If it is set to True, filelists will be downloaded during the standard metadata synchronization procedure. Additionally, specific commands can override this behavior during the runtime using the existing demands object in the DNF and so explicitly request loading the filelists metadata. This is used when a filename spec (other than rpm package filename) is passed as the command-line parameter.

Feedback

Benefit to Fedora

As DNF is integral to various infrastructure tasks like package building and installation, testing environment creation, and server integration tests, this change significantly reduces processing time and resource usage for these processes.

Scope

  • Proposal owners:
    • libdnf
      • Modify the Repo object to enable conditional filelists metadata download
      • Introduce a new main configuration option to set the default behavior
    • dnf
      • Introduce a new demand to enable specific commands to override filelists metadata download behavior
      • Handle demand and configuration option inputs to delegate filelists loading decision to libdnf
      • Implement filename pattern argument detection heuristics
  • Other developers:
    • The implementation work is isolated only to the mentioned two DNF stack components
    • There is no other development work related outside of the DNF stack
  • Release engineering: N/A (not needed for this Change)
  • Policies and guidelines:
    • Package maintainers need to comply with the Fedora packaging guidelines related to file dependencies specification (see here)
  • Trademark approval: N/A (not needed for this Change)
  • Alignment with Community Initiatives: N/A (no currently active initiatives)

Upgrade/compatibility impact

In general, applying these changes should not affect any existing user workflows and no additional manual changes are required. However, the absence of filelists might create an issue with packages that are not correctly packaged, f.e. from third-party repositories.

How To Test

User Experience

Large filelists could be over 200MB in size. It could take 1-2 minutes to download which is greatly slowing down the user experience.

For many operations the filelists metadata are not needed, so downloading them is wasting the resources. Without filelists being downloaded, DNF performance will be improved significantly, mainly regarding the network, CPU and disk space resources. Metadata download size will be reduced by about 60%. The improvement includes deployments of customer built RPMS to containers that have no need for filelists level dependencies.

Dependencies

Contingency Plan

  • Contingency mechanism: (What to do? Who will do it?) N/A (not a System Wide Change)
  • Contingency deadline: N/A (not a System Wide Change)
  • Blocks release? N/A (not a System Wide Change), Yes/No


Documentation

N/A (not a System Wide Change)

Release Notes

Retrieved from "https://fedoraproject.org/w/index.php?title=Changes/DNFConditionalFilelists&oldid=692152"