Latest revision as of 15:57, 31 October 2023

Default podman created images to zstd:chunked

This is a proposed Change for Fedora Linux.
This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee.

Summary

Modify podman push to default to creating images with zstd:chunked, allowing users to modify containers.conf to continue to use the legacy gzip image format.

Owner

Name: Daniel J Walsh
Email: dwalsh@redhat.com
Name: Giuseppe Scrivano
Email: giuseppe@redhat.com

Current status

Targeted release: Fedora Linux 40
Last updated: 2023-10-31
[<will be assigned by the Wrangler> devel thread]
FESCo issue: <will be assigned by the Wrangler>
Tracker bug: <will be assigned by the Wrangler>
Release notes tracker: <will be assigned by the Wrangler>

Detailed Description

The goal is to speed up the pulling of container images, via using zstd:chunked. zstd:chunked format has shown itself to speed up the pulling of images by as much as 90%. This change has been waiting for change for over 3 years, waiting for Docker to support the format.

Feedback

Benefit to Fedora

Fedora would be the first distribution to support building zstd:chunked format by default. Container engines (Podman, Buildah, Skopeo, CRI-O) which support the format natively would see a great speed up in pulling images. Other container engines including Docker and Containerd would see some speed up do the the compression algorythms of zstd, but not to the same degree as container engines with full support.

Scope

Proposal owners:

Other developers:

Release engineering: #Releng issue number

Policies and guidelines: N/A (not needed for this Change)

Trademark approval: N/A (not needed for this Change)

Alignment with Community Initiatives:

Upgrade/compatibility impact

How To Test

Build and push container images to container registries. Inspect them to check that their format is zstd:chunked. Pull the image back to the host. Now make a small change to the image and push it back to the registry Remove the changed image leaving the original image in place. Now pull the updated image again. You should see a big change in download speed.

Now attempt to pull the same image with Docker/Moby to make sure the image works with those container engines.

User Experience

Users should experience a nice improvement in download speeds of container images that were previously created with zstd:chunked.

Dependencies

Contingency Plan

If we find issues with zstd:chunked while turned on in Rawhide, we can easily reverse direction and go back to the original default gzip format.

Documentation

N/A (not a System Wide Change)

Release Notes

@@ Line 1: / Line 1: @@
-{{admon/important | Comments and Explanations | The page source contains comments providing guidance to fill out each section. They are invisible when viewing this page. To read it, choose the "view source" link.<br/> '''Copy the source to a ''new page'' before making changes!  DO NOT EDIT THIS TEMPLATE FOR YOUR CHANGE PROPOSAL.'''}}
-{{admon/tip | Guidance | For details on how to fill out this form, see the [https://docs.fedoraproject.org/en-US/program_management/changes_guide/ documentation].}}
-{{admon/tip | Report issues | To report an issue with this template, file an issue in the [https://pagure.io/fedora-pgm/pgm_docs pgm_docs repo].}}
 <!-- The actual name of your proposed change page should look something like: Changes/Your_Change_Proposal_Name.  This keeps all change proposals in the same namespace -->
-= Change Proposal Name <!-- The name of your change proposal --> =
+= Default podman created images to zstd:chunked =
 {{Change_Proposal_Banner}}
 == Summary ==
-<!-- A sentence or two summarizing what this change is and what it will do. This information is used for the overall changeset summary page for each release. Note that motivation for the change should be in the Benefit to Fedora section below, and this part should answer the question "What?" rather than "Why?". -->
+Modify podman push to default to creating images with zstd:chunked, allowing users to modify containers.conf to continue to use the legacy gzip image format.
 == Owner ==
@@ Line 19: / Line 13: @@
 This should link to your home wiki page so we know who you are.
 -->
-* Name: [[User:FASAcountName| Your Name]]
+* Name: [[User:dwalsh| Daniel J Walsh]]
 <!-- Include you email address that you can be reached should people want to contact you about helping with your change, status is requested, or technical issues need to be resolved. If the change proposal is owned by a SIG, please also add a primary contact person. -->
-* Email: <your email address so we can contact you, invite you to meetings, etc. Please provide your Bugzilla email address if it is different from your email in FAS>
+* Email: dwalsh@redhat.com
 <!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
 * FESCo shepherd: [[User:FASAccountName| Shehperd name]] <email address>
 -->
+* Name: [[User:giuseppe| Giuseppe Scrivano]]
+* Email: giuseppe@redhat.com
 == Current status ==
@@ Line 38: / Line 33: @@
 <!-- [[Category:SystemWideChange]] -->
-* Targeted release: [https://docs.fedoraproject.org/en-US/releases/f<VERSION>/ Fedora Linux <VERSION>]
+* Targeted release: [https://docs.fedoraproject.org/en-US/releases/f40/ Fedora Linux 40]
 * Last updated: <!-- this is an automatic macro — you don't need to change this line -->  {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}
 <!-- After the change proposal is accepted by FESCo, tracking bug is created in Bugzilla and linked to this page
@@ Line 53: / Line 48: @@
 == Detailed Description ==
 <!-- Expand on the summary, if appropriate.  A couple sentences suffices to explain the goal, but the more details you can provide the better. -->
+The goal is to speed up the pulling of container images, via using zstd:chunked.  zstd:chunked format has shown itself to speed up the pulling of images by as much as 90%. This change has been waiting for change for over 3 years, waiting for Docker to support the format.
 == Feedback ==
 <!-- Summarize the feedback from the community and address why you chose not to accept proposed alternatives. This section is optional for all change proposals but is strongly suggested. Incorporating feedback here as it is raised gives FESCo a clearer view of your proposal and leaves a good record for the future. If you get no feedback, that is useful to note in this section as well. For innovative or possibly controversial ideas, consider collecting feedback before you file the change proposal. -->
 == Benefit to Fedora ==
-<!-- What is the benefit to the distribution?  Will the software we generate be improved? How will the process of creating Fedora releases be improved?
+Fedora would be the first distribution to support building zstd:chunked format by default. Container engines (Podman, Buildah, Skopeo, CRI-O)  which support the format natively would see a great speed up in pulling images.  Other container engines including Docker and Containerd would see some speed up do the the compression algorythms of zstd, but not to the same degree as container engines with full support.
-      Be sure to include the following areas if relevant:
-      If this is a major capability update, what has changed?
-           For example: This change introduces Python 5 that runs without the Global Interpreter Lock and is fully multithreaded.
-      If this is a new functionality, what capabilities does it bring?
-           For example: This change allows package upgrades to be performed automatically and rolled-back at will.
-      Does this improve some specific package or set of packages?
-           For example: This change modifies a package to use a different language stack that reduces install size by removing dependencies.
-      Does this improve specific Spins or Editions?
-           For example: This change modifies the default install of Fedora Workstation to be more in line with the base install of Fedora Server.
-      Does this make the distribution more efficient?
-           For example: This change replaces thousands of individual %post scriptlets in packages with one script that runs at the end.
-      Is this an improvement to maintainer processes?
-           For example: Gating Fedora packages on automatic QA tests will make rawhide more stable and allow changes to be implemented more smoothly.
-      Is this an improvement targeted as specific contributors?
-           For example: Ensuring that a minimal set of tools required for contribution to Fedora are installed by default eases the onboarding of new contributors.
-     When a Change has multiple benefits, it's better to list them all.
-     Consider these Change pages from previous editions as inspiration:
-     https://fedoraproject.org/wiki/Changes/Annobin (low-level and technical, invisible to users)
-     https://fedoraproject.org/wiki/Changes/ParallelInstallableDebuginfo (low-level, but visible to advanced users)
-     https://fedoraproject.org/wiki/Changes/VirtualBox_Guest_Integration (primarily a UX change)
-     https://fedoraproject.org/wiki/Changes/NoMoreAlpha (an improvement to distro processes)
-     https://fedoraproject.org/wiki/Changes/perl5.26 (major upgrade to a popular software stack, visible to users of that stack)
--->
 == Scope ==
@@ Line 113: / Line 82: @@
 == How To Test ==
-<!-- This does not need to be a full-fledged document. Describe the dimensions of tests that this change implementation is expected to pass when it is done.  If it needs to be tested with different hardware or software configurations, indicate them.  The more specific you can be, the better the community testing can be.
+Build and push container images to container registries. Inspect them to check that their format is zstd:chunked.  Pull the image back to the host.  Now make a small change to the image and push it back to the registry
+Remove the changed image leaving the original image in place.  Now pull the updated image again.  You should see a big change in download speed.
-Remember that you are writing this how to for interested testers to use to check out your change implementation - documenting what you do for testing is OK, but it's much better to document what *I* can do to test your change.
-A good "how to test" should answer these four questions:
-. What special hardware / data / etc. is needed (if any)?
-. How do I prepare my system to test this change? What packages
-need to be installed, config files edited, etc.?
-. What specific actions do I perform to check that the change is
-working like it's supposed to?
-. What are the expected results of those actions?
--->
-<!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
+Now attempt to pull the same image with Docker/Moby to make sure the image works with those container engines.
 == User Experience ==
-<!-- If this change proposal is noticeable by users, how will their experiences change as a result?
+Users should experience a nice improvement in download speeds of container images that were previously created with zstd:chunked.
- This section partially overlaps with the Benefit to Fedora section above. This section should be primarily about the User Experience, written in a way that does not assume deep technical knowledge. More detailed technical description should be left for the Benefit to Fedora section.
- Describe what Users will see or notice, for example:
-  - Packages are compressed more efficiently, making downloads and upgrades faster by 10%.
-  - Kerberos tickets can be renewed automatically. Users will now have to authenticate less and become more productive. Credential management improvements mean a user can start their work day with a single sign on and not have to pause for reauthentication during their entire day.
- - Libreoffice is one of the most commonly installed applications on Fedora and it is now available by default to help users "hit the ground running".
- - Green has been scientifically proven to be the most relaxing color. The move to a default background color of green with green text will result in Fedora users being the most relaxed users of any operating system.
--->
 == Dependencies ==
@@ Line 150: / Line 98: @@
 == Contingency Plan ==
-<!-- If you cannot complete your feature by the final development freeze, what is the backup plan?  This might be as simple as "Revert the shipped configuration".  Or it might not (e.g. rebuilding a number of dependent packages).  If you feature is not completed in time we want to assure others that other parts of Fedora will not be in jeopardy.  -->
+If we find issues with zstd:chunked while turned on in Rawhide, we can easily reverse direction and go back to the original default gzip format.
-* Contingency mechanism: (What to do?  Who will do it?) N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-<!-- When is the last time the contingency mechanism can be put in place?  This will typically be the beta freeze. -->
-* Contingency deadline: N/A (not a System Wide Change)  <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
-<!-- Does finishing this feature block the release, or can we ship with the feature in incomplete state? -->
-* Blocks release? N/A (not a System Wide Change), Yes/No <!-- REQUIRED FOR SYSTEM WIDE CHANGES -->
 == Documentation ==

Search

Changes/zstd:chunked: Difference between revisions