From Fedora Project Wiki
mNo edit summary
(matching release notes with #515)
 
(50 intermediate revisions by 3 users not shown)
Line 1: Line 1:
= swap on ZRAM =
= swap on zram =


== Summary ==
== Summary ==


Swap is useful, except when slow.[1] ZRAM is a RAM disk that uses always-on compression [2]. It has a size assigned at create time, but the RAM usage is dynamically allocated and deallocated, on demand. This ZRAM block device behaves like any other, it can be formatted with a file system or mkswap, which is the intention with this change proposal.
Swap is useful, except when it's slow. zram is a RAM drive that uses compression. Create a swap-on-zram during start-up. And no longer use swap partitions by default.
 
There is more than one change indicated in this proposal. Each is opt-in (owner assumes editions/spins are to be excluded unless they ask to be included):
 
1. Include systemd rust-zram-generator[3]. This does not enable swap-on-ZRAM. It only makes the generator available.</br >
2. Include a default zram-generator configuration. If present, it enables swap-on-ZRAM during startup.</br >
3. Do not create swap partition/LV for default installations.
 
 
[1]</br >
There is a tl;dr section at the top. Highly recommend reading the whole article.</br >
In defence of swap: common misconceptions</br >
https://chrisdown.name/2018/01/02/in-defence-of-swap.html
 
[2]</br >
https://www.kernel.org/doc/Documentation/blockdev/zram.txt
 
[3]</br >
https://github.com/systemd/zram-generator
 




== Owner ==
== Owner ==
* Name: [[User:chrismurphy| Chris Murphy]]
* Name: [[User:chrismurphy| Chris Murphy]]
* Email: chrismurphy@fedoraproject.org
* Email: bugzilla@colorremedies.com
<!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
<!--- UNCOMMENT only for Changes with assigned Shepherd (by FESCo)
* FESCo shepherd: [[User:FASAccountName| Shehperd name]] <email address>
* FESCo shepherd: [[User:FASAccountName| Shehperd name]] <email address>
Line 35: Line 16:
* Responsible WG:
* Responsible WG:
-->
-->


== Current status ==
== Current status ==
[[Category:ChangePageIncomplete]]
[[Category:ChangeAcceptedF33]]
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
<!-- remove Category:ChangePageIncomplete and change it to Category:ChangeReadyForWrangler -->
[[Category:SystemWideChange]]
[[Category:SystemWideChange]]


* Targeted release: [[Releases/33 | Fedora 33 ]]  
* Targeted release: [[Releases/33 | Fedora 33 ]]  
* Last updated: <!-- this is an automatic macro — you don't need to change this line -->  {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}  
* Last updated: <!-- this is an automatic macro — you don't need to change this line -->  {{REVISIONYEAR}}-{{REVISIONMONTH}}-{{REVISIONDAY2}}  
* FESCo issue: <will be assigned by the Wrangler>
* FESCo issue: [https://pagure.io/fesco/issue/2408 #2408]
* Tracker bug: <will be assigned by the Wrangler>
* Tracker bug: [https://bugzilla.redhat.com/show_bug.cgi?id=1850218 #1850218]
* Release notes tracker: <will be assigned by the Wrangler>
* Release notes tracker: [https://pagure.io/fedora-docs/release-notes/issue/515 #515]


== Detailed Description ==
== Detailed Description ==


==== Basic function: ====
==== zram Basic function ====
 
The zram† device, typically <span style=color:brown>/dev/zram0</span>, has a size set at create time during early boot, by zram-generator† per its configuration file. The memory used is not preallocated. It's dynamically allocated and deallocated, on demand. Due to compression, a full <span style=color:brown>/dev/zram0</span> uses half as much memory as its size.


The system will use RAM normally up until it's full, and then start paging out to the swap-on-ZRAM device, just as if it were a real swap. But, there is no free lunch. The ZRAM driver starts to allocate memory at roughly 1/2 the rate of page outs, due to compression. This means swap is not as effective at page eviction, the rate is ~50% instead of 100%. But it is orders of magnitude faster that disk based swap.
The <span style=color:brown>/dev/zram0</span> behaves like any other block device. It can be formatted with a file system, or mkswap, which is the intention with this change proposal.  


ZRAM has about 0.1% overhead or ~1MiB/1GiB. If the workload never touches swap, living entirely inside RAM, the overhead is the sole cost, there is no preallocation of RAM.
The system will use RAM normally up until it's full, and then start paging out to swap-on-zram, same as a conventional swap-on-drive. The zram driver starts to allocate memory at roughly 1/2 the rate of page outs, due to compression. But, there is no free lunch. This means swap-on-zram is not as effective at page eviction as swap-on-drive, the eviction rate is ~50% instead of 100%. But it is at least an order of magnitude faster than drive based swap.


==== Default configuration: ====
zram has about 0.1% overhead or ~1MiB/1GiB. If the workload never touches swap, this overhead is the sole cost. In practice when not used at all, feature owner has experienced ~0.04% overhead.


Always create a ZRAM device regardless, of RAM size, using a ZRAM to RAM ratio of 1:2, and capped to 4GiB [4], with a higher than typical swap priority [5].
Example: A system has 16 GiB RAM. The proposed defaults suggest the <span style=color:brown>/dev/zram0</span> device will be 4 GiB. If the workload completely fills up swap with 4 GiB of anonymous pages, what's happened? The <span style=color:red>zramctl</span> command will display the true compression ratio. If 2:1 is really obtained, it means 4GiB swap data is compressed to 2GiB. Therefore 2GiB is the actual RAM usage, and is also the net effective eviction. i.e. 4 GiB anonymous pages are evicted, but are then compressed and pinned into 2 GiB RAM, for a net memory savings of 2 GiB.


†</br >
[https://www.kernel.org/doc/Documentation/blockdev/zram.txt kernel.org zram.txt]


[4]</br >
[https://github.com/systemd/zram-generator Github zram-generator project]
RFE: should be able to set a cap on zram device size</br >
https://github.com/systemd/zram-generator/issues/10


[5]</br >
RFE: should set priority #8 </br >
https://github.com/systemd/zram-generator/issues/8


These values seem reasonable, based on prior work. Anaconda has two examples in which it sets swap size to 50% RAM: the no hibernation case, common outside x86; and it's own current swap-on-ZRAM implementation. Fedora IoT's implementation uses a ZRAM to RAM ratio of 1:2, or 50% as well.  
==== Overview of the Feature ====
 
Using swap is a good idea†, but no one likes it when it's slow. Anaconda and Fedora IoT have been using swap-on-zram by default for years. This builds on their prior effort.
 
 
There are three components to the change:
 
# Install `zram-generator` package†. This does not enable swap-on-zram, it only makes the generator available.</br >
# Install `zram-generator-defaults` package, which provides a default configuration. When present, swap-on-zram is set-up during startup.</br >
# Do not create swap partition/LV with default installations.
 
This proposal aims to apply all three, for all Fedora editions and spins, by default.
 
It further aims to apply the first two, for upgrades and custom installations.


==== In the summary, three changes are listed. What does the opt in look like for these? What's the benefit of this apparent complexity? ====
It might be useful to only make the generator available (1), should an edition/spin wish to opt out, or as a fallback if applying the feature to upgrades fails to withstand scrutiny.


(1) only = generator present, users can enable by creating a configuration file. No other changes. Ideally FESCo approves this Fedora wide so that the generator is available everywhere without exception. Makes it easier to converge on to reduce user confusion,
†</br >
There is a tl;dr section at the top. Highly recommend reading the whole article. [https://chrisdown.name/2018/01/02/in-defence-of-swap.html In defence of swap: common misconceptions]


(1) + (2) = swap-on-ZRAM is enabled, and with a higher priority than default for swap-on-drive. Both co-exist, but swap-on-ZRAM is favored first. Hibernation is still possible if the swap-on-drive partition is big enough and all other requirements are met.
==== Default zram device configuration: ====


(1) + (2) + (3) = swap-on-ZRAM is enabled, no disk-based swap present. Fedora Workstation edition plans on doing this (pending test day results and feedback). [6]
During startup, create a zram device <span style=color:brown>/dev/zram0</span>, with a size equal to 50% RAM, but capped† to 4 GiB, and with a higher than typical swap priority†.


These values seem reasonably conservative, and are based on prior work in Fedora. Anaconda sets swap-on-drive sized to 50% RAM in the no hibernation case, common outside x86. Fedora IoT's implementation also sets swap-on-zram size to 50% RAM.


[6]</br >
† <strike>[https://github.com/systemd/zram-generator/issues/10 RFE: should be able to set a cap on zram device size #10]</strike> (DONE)<br>
Pending test day results, Fedora Workstation Edition anticipates proceeding with full opt in: swap-on-ZRAM is enabled. New default installs will not have a swap-on-disk present. If the user opts to create swap via Custom partitioning at installation time, the installation will have two swaps: swap-on-ZRAM, swap-on-disk. The swap-on-ZRAM will have higher priority, thus being favored over disk based swap. The kernel is smart enough to know it can't hibernation to a ZRAM device, and will instead use disk based swap. This works reliably in testing so far, and we'll want to thoroughly beat it up during the test day.
<strike>[https://github.com/systemd/zram-generator/issues/8 RFE: should set priority #8]</strike> (DONE)


==== Default installer behavior  ====


The installer is currently responsible for creating a swap-on-drive device. This will be dropped. The zram-generator + configuration file will trigger the setup and activation of swap-on-zram. This means hibernation isn't possible, even on systems that could support it.


==== Test Day: ====
Please see [https://pagure.io/fedora-workstation/blob/master/f/hibernationstatus.md Supporting hibernation in Workstation edition] for much more detailed information, including why it's increasingly likely hibernation isn't possible anyway, and a path to improving hibernation support.


Change owner recommends all editions/spins opt in and participate in the soon to be scheduled test day. The defaults will be conservative. e.g. 50% RAM used for the ZRAM device with a cap of 4GiB. Test day will help fine tune this.


==== Custom/Advance partitioning installer behavior ====


The user can add swap using Custom partitioning at install time. This is swap-on-drive. And the installer will also include the <span style=color:red> resume=UUID </span> kernel parameter for this swap device. No change in behavior here.


== Feedback ==
Since swap-on-zram is still enabled by default, there will be two swaps: swap-on-zram, and swap-on-drive. The swap-on-zram will have higher priority, thus being favored over drive based swap. The kernel is smart enough to know it can't hibernate to a zram device, and will instead use drive based swap.


==== Why not zswap? ====


Zswap is a similar idea, similar "z" affection, but with a totally different implementation. It is swap specific, uses a RAM cache, and requires a conventional swap partition existing already. It might be true certain workloads are better suited for using zswap. But swap-on-ZRAM depends only on volatile storage. This is simpler and it's more secure. Whereas zswap "spilling over" into the real swap on disk can leak user data if that swap device isn't encrypted. This is certainly a valid future feature for a new generator, or possibly zram-generator could be extended to include support for zswap via the configuration file.
==== How can it be disabled? ====


Immediately:</br >
<span style=color:red>sudo systemctl stop swap-create@zram0</span>


https://www.kernel.org/doc/Documentation/vm/zswap.txt
Permanently:</br >
<span style=color:red>sudo touch /etc/systemd/zram-generator.conf</span>
or
<span style=color:red>sudo dnf remove zram-generator-defaults</span>


== Feedback ==
   
   
==== You're enabling it on upgrades? ====
==== You're enabling it on upgrades? ====


That's the current plan. There are some difficulties with this right now in Fedora, needing to use Supplements to cause new things to be dragged in on upgrades. As a technical matter, feature owner is confident it will improve the experience of users who have configured their systems with no swap. As a non-technical matter, it's recognized that (a) hey pal, you're messing with my customizations, not cool! and (b) swap always sucks I don't care if it has a 'Z' in the name! The dilemma is that not applying it to upgrades fragments the Fedora user base, and the overall experience people are having, and giving feedback on to make Fedora better. So all of this has to be balanced out.
That's the current plan. As a technical matter, feature owner is confident this feature will improve the experience of all users regardless of configuration. As a non-technical matter, it's recognized that (a) ''hey pal, you're messing with my customizations, not cool!'' and (b) ''swap always stinks, I don't care if it has a 'Z' in the name!'' may need more convincing.
 
There are possible risks.
 
* Workloads that expect full use of memory, and depend on 100% page eviction. These may run slower if they really need full use of memory, but some memory is used for the zram device instead. Such workloads might favor zswap.
 
* Workloads with low compressible pages. In the worst case, this means unnecessary work merely moving pages around.


* Workloads with memory full, and hibernation. Hibernation is already stressful to memory-management subsystem and prone to bailing out in such cases. The swap-on-zram will be favored for evictions in the attempt to free memory to create the hibernation image. It could increase instances of hibernation entry failure. This isn't a crash, it just means the attempt doesn't succeed, and the system resumes operation instead of hibernating.


While possible, it's difficult to estimate their probability. But this is a significant consideration in the conservative default zram size. Users can easily increase zram size as needed for their use case, simply by editing <span style=color:red>/etc/systemd/zram-generator.conf</span> and the change takes effect at next boot.


==== Why systemd zram-generator? ====
==== Why systemd zram-generator? ====


It's the most upstream implementation to date, and leverages existing systemd infrastructure setup the ZRAM block device, format it as swap, and swapon - all during early boot. There are also systemd-fstab-generator
It's the most upstream implementation to date, is fast and lightweight. The zram-generator uses existing systemd infrastructure to setup the zram block device, format it as swap, and swapon - all during early boot. It's very similar in behavior to fstab-generator, gpt-auto-generator, and cryptsetup-generator†.
 
Converging on one implementation avoids user confusion. And while the alternatives are nice and work fine, a systemd generator is particularly well suited for this use case compared to a systemd service unit.†
 
Also, it's an reference implementation of a system generator written in Rust.
 
†</br >
[https://www.freedesktop.org/software/systemd/man/systemd.generator.html freedesktop.org About systemd generators.]</br >
[https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/TCY534JPIMZ3OXM5Q5E2ZH5PSAKQNGP7/ devel@ ''Re: swap-on-zram by default'' Zbigniew Jędrzejewski-Szmek, systemd zram-generator author/maintainer]
 
 
==== Why not a bigger zram device? ====
 
The main idea of being conservative is to address concerns about upgrades. It's possible some workloads will have less compressible data. Hence, not going with <span style=color:brown>/dev/zram0</span> sized to 100% of RAM at this time. Even a <span style=color:brown>/dev/zram0</span> of 200% RAM is not unreasonable *if* the compression ratio is at least 2:1. However, it's possible a system can get "stuck" in a kind of swap thrashing similar to conventional swap-on-drive, except it's CPU and memory bound, rather than IO bound. Feature owner thinks it's better to just oom, instead of getting overly aggressive with the zram device size.
 
Conversely it's possible to be too conservative with the size, and result in more instances of OOM kill. If applying the feature to upgrades is rejected, it's probably reasonable to increase the cap to ~8GiB. Of course more feedback and testing is needed, and it will be taken into consideration.
 
Note that the kernel zram doc says an excessively sized zram device does come with overhead. Users's can increase the size easily post-install, a capability they don't easily have with swap-on-drive. The goal for Fedora 33 is a default that's useful and safe for the vast majority of use cases.
 


Also the idea is to converge on one particular implementation to avoid user confusion. And while the alternatives are nice and work fine, using a systemd generator rather than a systemd service unit is ideal.</br >
==== Why not zswap? ====
https://www.freedesktop.org/software/systemd/man/systemd.generator.html</br >
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/TCY534JPIMZ3OXM5Q5E2ZH5PSAKQNGP7/


Zswap† is a similar idea, speed up swapping, but with a different implementation. It needs disk based swap, and uses a compressed memory cache to hold onto recently used pages, where less recently used pages are evicted to to swap.


Swap-on-zram depends only on volatile storage. This is simpler and more secure. Whereas zswap eviction of pages into swap-on-drive can leak user data. Some workloads may do better with zswap, and it's a valid future feature for this generator. One idea is that the generator could favor setting up zswap when swap-on-drive already exists; and fallback to swap-on-zram?


==== Why not a bigger ZRAM device? ====
===== What if I'm already using zswap? =====


It's possible some workloads will have data that doesn't compress well. Hence not going with a 1:1 ZRAM to RAM ratio. Even a 2:1 ratio is not unreasonable *if* the compression ratio is 2:1. However, it's possible a system can actually get "wedged in" to a kind of swap thrashing very similar to conventional swap-on-disk, except it becomes CPU and memory bound, rather than IO bound. Feature owner thinks it's better to just oom than get overly aggressive with the ZRAM device size. And this feature is already well tested with earlyoom used by default on Fedora Workstation since Fedora 32.
Feature owner recommends disabling the swaponzram feature, found in this proposal. [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/LORCBGD4I67LRFM3FCFYUHFQZJTR5K6A/ More information on devel@ reponse.]


†</br >
[https://www.kernel.org/doc/Documentation/vm/zswap.txt kernel.org zswap.txt]




Line 128: Line 157:


* significantly improves system responsiveness, especially when swap is under pressure;
* significantly improves system responsiveness, especially when swap is under pressure;
* complements on-going resource control work;  
* more secure, user data leaks into swap are on volatile media;
* without swap-on-drive, there's better utilization of a limited resource: benefit of swap without the drive space consumption;
* complements on-going resource control work, including earlyoom;
* further reduces the time to out-of-memory kill, when workloads exceed limits;
* further reduces the time to out-of-memory kill, when workloads exceed limits;
* improves both the "no swap" and "existing swap" setups;
* improves performance for both "no swap" and "existing swap" setups;




Line 137: Line 168:


* Proposal owners:  
* Proposal owners:  
** add zram-generator package to comps for the editions/spins opting in
** add zram-generator package to comps and kickstarts as appropriate
** obsolete zram package (used by Fedora IoT)
** means of per edition/spin configurations, if needed
** means of per edition/spin configurations, if needed
** coordinate a test day
** test day, see https://pagure.io/fedora-qa/issue/632


* Other developers:
* Other developers:
**RFE's for zram-generator: users are not worse off if they don't happen</br >
**Anaconda are agreeable to deprecating their built-in implementation in favor of swap-on-zram
https://github.com/systemd/zram-generator/issues/10</br >
**RFE's for zram-generator: users are not worse off if they don't happen. Open request for help, to make it possible. It's much appreciated.</br >
https://github.com/systemd/zram-generator/issues/8
<strike>[https://github.com/systemd/zram-generator/issues/10 RFE: should be able to set a cap on zram device size #10]</strike> (DONE)</br >
<strike>[https://github.com/systemd/zram-generator/issues/8 RFE: should set priority #8]</strike> (DONE)


** Anaconda is agreeable to deprecating their built-in implementation in favor of swap-on-ZRAM
* Release engineering: [https://pagure.io/releng/issues/9495 #9495]


* Policies and guidelines: N/A


* Trademark approval: N/A


== Upgrade/compatibility impact ==
== Upgrade/compatibility impact ==


If all editions/spins opt in, add Supplements:fedora-release to zram-generator to pull it in on upgrades.
Add <code>Obsoletes: zram < 0.4-2</code> to <span style=color:blue>zram-generator-defaults</span>. This means only systems that have <span style=color:blue>zram</span>, will get <span style=color:blue>zram-generator-defaults</span>. And it means they will have swap-on-zram enabled post-upgrade, whether or not it was previously enabled.  


Existing systems with no swap will have swap-on-ZRAM enabled.
Fedora Workstation has included <span style=color:blue>zram</span> since July 2019 (Fedora 31) by default. Any clean installed systems from that point will automatically be upgraded to this feature.
Existing systems with conventional swap-on-disk, will also have swap-on-ZRAM enabled (two swap devices), with higher priority for the ZRAM device. Existing swap-on-disk will not be removed.


Fedora IoT has included <span style=color:blue>zram</span> from the beginning. All systems will automatically get this feature upon upgrade.




Line 163: Line 198:
Any hardware. Any version of Fedora.
Any hardware. Any version of Fedora.


1. dnf install zram-generator
# dnf install zram-generator zram-generator-defaults
2. cp /usr/share/doc/zram-generator/zram-generator.conf.example /etc/systemd/zram-generator.conf
# Reboot
3. Edit the configuration
# Check that swap is on a zram device: zramctl, swapon
4. Reboot
# Detailed check: journalctl -b -o short-monotonic --grep 'swap|zram'
5. Check that swap is on a ZRAM device
# Check that priority is higher than existing swap if two or more are listed.
zramctl
 
swapon
Feel free to run your usual workloads more aggressively or in parallel. Suspend-to-RAM and suspend-to-drive are expected to continue to work too (or at least hit all the same bugs as without zram being used).
6. Detailed check: journalctl -b -o short-monotonic | grep 'swap\|zram'
 
7. Check that priority is higher than existing swap if two or more are listed.
Also, you can see the actual compression ratio achieved with the following command:</br >
<span style=color:red> zramctl </span>
 
 
==== Test Day ====
 
[https://fedoraproject.org/wiki/Test_Day:F33_SwapOnZRAM QA: SwapOnzram Test Day] to discover edge cases, and tweak the default configuration if necessary to establish a good one-size-fits all approach.


== User Experience ==
== User Experience ==


The user won't notice anything. If their usual workload causes them to dread swap thrashing, they'll be surprised this doesn't happen. The user might get curious if they don't find a swap entry in /etc/fstab. Or they might get curious if they 'swapon' and see swap pointing to /dev/zram0 instead of a disk partition or LV.
The user won't notice anything displeasing. If their usual workload causes them to dread swap thrashing, they'll be surprised that thrashing doesn't happen. The user might get curious if they don't find a swap entry in /etc/fstab. Or if they 'swapon' and see swap pointing to <span style=color:brown>/dev/zram0</span> instead of a drive partition or LV.
 


== Dependencies ==
== Dependencies ==
Line 185: Line 226:
== Contingency Plan ==
== Contingency Plan ==


* Contingency mechanism: Don't ship the generator = big hammer but easy. Also possible to ship the generator, but only selectively ship configuration files = scalpel, pretty easy.
* Contingency mechanism: Don't ship the generator = big hammer, but easy. Preferable to ship the generator, but only selectively ship configuration files = scalpel, pretty easy.
* Contingency deadline: Beta freeze
* Contingency deadline: Beta freeze
* Blocks release? No.
* Blocks release? No.
Line 191: Line 232:




<code>man 8 zram-generator</code><br />
<code>man 5 zram-generator.conf</code>


== Documentation ==
(Check out the ASCII art!)
 
Fedora could consider adding a hint in an /etc/fstab comment? There is no man page for this, and the documentation is also minimal besides what's in this feature proposal. So it's an open question how the user should get more information on how to configure and tweak it. But then, they don't have that for swap today either. They just have institutional knowledge.
 
Hence, a strong test day with a lot of people and press coverage of the feature might help spread the word for institutional knowledge changes coming.
 
Ideas welcome.
 
 


== Release Notes ==
== Release Notes ==


Pending feedback and test day.
A swap partition is not created by default at installation time. Instead, a zram device is created, and swap enabled on it during start-up. zram is a RAM drive that uses compression. See `man zram-generator` for a brief overview of its function. The swap-on-zram feature can be disabled with `sudo touch /etc/systemd/zram-generator.conf` and reenabled by removing this file, and customized by editing it. See `man zram-generator.conf` for configuration information, including a description of the default configuration plus ASCII art.

Latest revision as of 02:59, 13 October 2020

swap on zram

Summary

Swap is useful, except when it's slow. zram is a RAM drive that uses compression. Create a swap-on-zram during start-up. And no longer use swap partitions by default.


Owner


Current status

Detailed Description

zram Basic function

The zram† device, typically /dev/zram0, has a size set at create time during early boot, by zram-generator† per its configuration file. The memory used is not preallocated. It's dynamically allocated and deallocated, on demand. Due to compression, a full /dev/zram0 uses half as much memory as its size.

The /dev/zram0 behaves like any other block device. It can be formatted with a file system, or mkswap, which is the intention with this change proposal.

The system will use RAM normally up until it's full, and then start paging out to swap-on-zram, same as a conventional swap-on-drive. The zram driver starts to allocate memory at roughly 1/2 the rate of page outs, due to compression. But, there is no free lunch. This means swap-on-zram is not as effective at page eviction as swap-on-drive, the eviction rate is ~50% instead of 100%. But it is at least an order of magnitude faster than drive based swap.

zram has about 0.1% overhead or ~1MiB/1GiB. If the workload never touches swap, this overhead is the sole cost. In practice when not used at all, feature owner has experienced ~0.04% overhead.

Example: A system has 16 GiB RAM. The proposed defaults suggest the /dev/zram0 device will be 4 GiB. If the workload completely fills up swap with 4 GiB of anonymous pages, what's happened? The zramctl command will display the true compression ratio. If 2:1 is really obtained, it means 4GiB swap data is compressed to 2GiB. Therefore 2GiB is the actual RAM usage, and is also the net effective eviction. i.e. 4 GiB anonymous pages are evicted, but are then compressed and pinned into 2 GiB RAM, for a net memory savings of 2 GiB.


kernel.org zram.txt

Github zram-generator project


Overview of the Feature

Using swap is a good idea†, but no one likes it when it's slow. Anaconda and Fedora IoT have been using swap-on-zram by default for years. This builds on their prior effort.


There are three components to the change:

  1. Install zram-generator package†. This does not enable swap-on-zram, it only makes the generator available.
  2. Install zram-generator-defaults package, which provides a default configuration. When present, swap-on-zram is set-up during startup.
  3. Do not create swap partition/LV with default installations.

This proposal aims to apply all three, for all Fedora editions and spins, by default.

It further aims to apply the first two, for upgrades and custom installations.

It might be useful to only make the generator available (1), should an edition/spin wish to opt out, or as a fallback if applying the feature to upgrades fails to withstand scrutiny.


There is a tl;dr section at the top. Highly recommend reading the whole article. In defence of swap: common misconceptions

Default zram device configuration:

During startup, create a zram device /dev/zram0, with a size equal to 50% RAM, but capped† to 4 GiB, and with a higher than typical swap priority†.

These values seem reasonably conservative, and are based on prior work in Fedora. Anaconda sets swap-on-drive sized to 50% RAM in the no hibernation case, common outside x86. Fedora IoT's implementation also sets swap-on-zram size to 50% RAM.

RFE: should be able to set a cap on zram device size #10 (DONE)
RFE: should set priority #8 (DONE)

Default installer behavior

The installer is currently responsible for creating a swap-on-drive device. This will be dropped. The zram-generator + configuration file will trigger the setup and activation of swap-on-zram. This means hibernation isn't possible, even on systems that could support it.

Please see Supporting hibernation in Workstation edition for much more detailed information, including why it's increasingly likely hibernation isn't possible anyway, and a path to improving hibernation support.


Custom/Advance partitioning installer behavior

The user can add swap using Custom partitioning at install time. This is swap-on-drive. And the installer will also include the resume=UUID kernel parameter for this swap device. No change in behavior here.

Since swap-on-zram is still enabled by default, there will be two swaps: swap-on-zram, and swap-on-drive. The swap-on-zram will have higher priority, thus being favored over drive based swap. The kernel is smart enough to know it can't hibernate to a zram device, and will instead use drive based swap.


How can it be disabled?

Immediately:
sudo systemctl stop swap-create@zram0

Permanently:
sudo touch /etc/systemd/zram-generator.conf or sudo dnf remove zram-generator-defaults

Feedback

You're enabling it on upgrades?

That's the current plan. As a technical matter, feature owner is confident this feature will improve the experience of all users regardless of configuration. As a non-technical matter, it's recognized that (a) hey pal, you're messing with my customizations, not cool! and (b) swap always stinks, I don't care if it has a 'Z' in the name! may need more convincing.

There are possible risks.

  • Workloads that expect full use of memory, and depend on 100% page eviction. These may run slower if they really need full use of memory, but some memory is used for the zram device instead. Such workloads might favor zswap.
  • Workloads with low compressible pages. In the worst case, this means unnecessary work merely moving pages around.
  • Workloads with memory full, and hibernation. Hibernation is already stressful to memory-management subsystem and prone to bailing out in such cases. The swap-on-zram will be favored for evictions in the attempt to free memory to create the hibernation image. It could increase instances of hibernation entry failure. This isn't a crash, it just means the attempt doesn't succeed, and the system resumes operation instead of hibernating.

While possible, it's difficult to estimate their probability. But this is a significant consideration in the conservative default zram size. Users can easily increase zram size as needed for their use case, simply by editing /etc/systemd/zram-generator.conf and the change takes effect at next boot.

Why systemd zram-generator?

It's the most upstream implementation to date, is fast and lightweight. The zram-generator uses existing systemd infrastructure to setup the zram block device, format it as swap, and swapon - all during early boot. It's very similar in behavior to fstab-generator, gpt-auto-generator, and cryptsetup-generator†.

Converging on one implementation avoids user confusion. And while the alternatives are nice and work fine, a systemd generator is particularly well suited for this use case compared to a systemd service unit.†

Also, it's an reference implementation of a system generator written in Rust.


freedesktop.org About systemd generators.
devel@ Re: swap-on-zram by default Zbigniew Jędrzejewski-Szmek, systemd zram-generator author/maintainer


Why not a bigger zram device?

The main idea of being conservative is to address concerns about upgrades. It's possible some workloads will have less compressible data. Hence, not going with /dev/zram0 sized to 100% of RAM at this time. Even a /dev/zram0 of 200% RAM is not unreasonable *if* the compression ratio is at least 2:1. However, it's possible a system can get "stuck" in a kind of swap thrashing similar to conventional swap-on-drive, except it's CPU and memory bound, rather than IO bound. Feature owner thinks it's better to just oom, instead of getting overly aggressive with the zram device size.

Conversely it's possible to be too conservative with the size, and result in more instances of OOM kill. If applying the feature to upgrades is rejected, it's probably reasonable to increase the cap to ~8GiB. Of course more feedback and testing is needed, and it will be taken into consideration.

Note that the kernel zram doc says an excessively sized zram device does come with overhead. Users's can increase the size easily post-install, a capability they don't easily have with swap-on-drive. The goal for Fedora 33 is a default that's useful and safe for the vast majority of use cases.


Why not zswap?

Zswap† is a similar idea, speed up swapping, but with a different implementation. It needs disk based swap, and uses a compressed memory cache to hold onto recently used pages, where less recently used pages are evicted to to swap.

Swap-on-zram depends only on volatile storage. This is simpler and more secure. Whereas zswap eviction of pages into swap-on-drive can leak user data. Some workloads may do better with zswap, and it's a valid future feature for this generator. One idea is that the generator could favor setting up zswap when swap-on-drive already exists; and fallback to swap-on-zram?

What if I'm already using zswap?

Feature owner recommends disabling the swaponzram feature, found in this proposal. More information on devel@ reponse.


kernel.org zswap.txt


Benefit to Fedora

  • significantly improves system responsiveness, especially when swap is under pressure;
  • more secure, user data leaks into swap are on volatile media;
  • without swap-on-drive, there's better utilization of a limited resource: benefit of swap without the drive space consumption;
  • complements on-going resource control work, including earlyoom;
  • further reduces the time to out-of-memory kill, when workloads exceed limits;
  • improves performance for both "no swap" and "existing swap" setups;


Scope

  • Proposal owners:
    • add zram-generator package to comps and kickstarts as appropriate
    • obsolete zram package (used by Fedora IoT)
    • means of per edition/spin configurations, if needed
    • test day, see https://pagure.io/fedora-qa/issue/632
  • Other developers:
    • Anaconda are agreeable to deprecating their built-in implementation in favor of swap-on-zram
    • RFE's for zram-generator: users are not worse off if they don't happen. Open request for help, to make it possible. It's much appreciated.

RFE: should be able to set a cap on zram device size #10 (DONE)
RFE: should set priority #8 (DONE)

  • Release engineering: #9495
  • Policies and guidelines: N/A
  • Trademark approval: N/A

Upgrade/compatibility impact

Add Obsoletes: zram < 0.4-2 to zram-generator-defaults. This means only systems that have zram, will get zram-generator-defaults. And it means they will have swap-on-zram enabled post-upgrade, whether or not it was previously enabled.

Fedora Workstation has included zram since July 2019 (Fedora 31) by default. Any clean installed systems from that point will automatically be upgraded to this feature.

Fedora IoT has included zram from the beginning. All systems will automatically get this feature upon upgrade.


How To Test

Any hardware. Any version of Fedora.

  1. dnf install zram-generator zram-generator-defaults
  2. Reboot
  3. Check that swap is on a zram device: zramctl, swapon
  4. Detailed check: journalctl -b -o short-monotonic --grep 'swap|zram'
  5. Check that priority is higher than existing swap if two or more are listed.

Feel free to run your usual workloads more aggressively or in parallel. Suspend-to-RAM and suspend-to-drive are expected to continue to work too (or at least hit all the same bugs as without zram being used).

Also, you can see the actual compression ratio achieved with the following command:
zramctl


Test Day

QA: SwapOnzram Test Day to discover edge cases, and tweak the default configuration if necessary to establish a good one-size-fits all approach.

User Experience

The user won't notice anything displeasing. If their usual workload causes them to dread swap thrashing, they'll be surprised that thrashing doesn't happen. The user might get curious if they don't find a swap entry in /etc/fstab. Or if they 'swapon' and see swap pointing to /dev/zram0 instead of a drive partition or LV.


Dependencies

N/A


Contingency Plan

  • Contingency mechanism: Don't ship the generator = big hammer, but easy. Preferable to ship the generator, but only selectively ship configuration files = scalpel, pretty easy.
  • Contingency deadline: Beta freeze
  • Blocks release? No.
  • Blocks product? No.


man 8 zram-generator
man 5 zram-generator.conf

(Check out the ASCII art!)

Release Notes

A swap partition is not created by default at installation time. Instead, a zram device is created, and swap enabled on it during start-up. zram is a RAM drive that uses compression. See man zram-generator for a brief overview of its function. The swap-on-zram feature can be disabled with sudo touch /etc/systemd/zram-generator.conf and reenabled by removing this file, and customized by editing it. See man zram-generator.conf for configuration information, including a description of the default configuration plus ASCII art.