Changes/BtrfsByDefault/CompressionLevelAnalysis

Simple Analysis of btrfs zstd compression level

Workstation root fs disk space savings

tl;dr;

For an installed root directory of Fedora 32, zstd:1 yields a 40% storage savings as compared to uncompressed ext4. zstd:9 yields a 43% storage savings.

Test Steps

1. Obtain an “installed.dir” by

   1. Prep the image
       1. truncate -s 64G installed.raw
   2. Booting the live cd (in the case F32) in a qemu-kvm 
       1. qemu-kvm -drive file=installed.raw -cdrom $ISO -m 1G
   3. Installing it in the graphical installer
       1. I used regular partitions, and did not create a new one for /home
   4. Shutting down the qemu instance3105440
   5. Mount it to the directory
       1. losetup -Pf installed.raw
       2. mkdir -p installed.dir
       3. mount /dev/loop1p2 installed.dir

2. Run the script below

#!/bin/sh

set -e

SIZE=16G

if [ $EUID -ne 0 ] ; then
    echo "Needs to be root" >&2
    exit 2
fi

for compress_level in 1 3 9 ; do
    raw=zstd${compress_level}.raw
    truncate -s $SIZE $raw
    mkfs.btrfs -f $raw
    dir=zstd${compress_level}.dir
    mkdir -p $dir
    mount -o compress=zstd:${compress_level} $raw $dir
    perf record -g -o perf${compress_level}.data bash -c "cp -r installed.dir/* $dir ; sync $dir"
done
</code>

=== Results ===

zstd:1 - 40% savings
zstd:3 - 41% savings
zstd:9 - 43% savings

This is the result from df. Ignore Use% because the installed directory is a different size.

<code>
Filesystem                              1K-blocks     Used Available Use% Mounted on
/dev/loop1p3                             61665068  5458980  53043976  10% /scratch/installed.dir
/dev/loop0                               16777216  3257192  13356936  20% /scratch/zstd1.dir
/dev/loop2                               16777216  3197668  13415468  20% /scratch/zstd3.dir
/dev/loop3                               16777216  3105440  13502704  19% /scratch/zstd9.dir

/dev/urandom copy cpu cost

• • note: this test is a lot less scientific than the other one, because it’ll be CPU dependent and I had other stuff going on (chrome was open, etc). It’s likely still a reasonable proxy for CPU impact though.**

tl;dr;

zstd:1 added 9% and zstd:9 added 15% of system time to the baseline.

Test Steps

1. Get a 2GB blah file that will compress very poorly

   1. dd if=/dev/urandom of=blah bs=256K count=8192

2. Run the script

#!/bin/sh

set -e

SIZE=16G

if [ $EUID -ne 0 ] ; then
    echo "Needs to be root" >&2
    exit 2
fi

for compress_level in 0 1 3 9 ; do
    raw=zstd${compress_level}.raw
    truncate -s $SIZE $raw    #perf record -g -o perf${compress_level}.data bash -c    #perf record -g -o perf${compress_level}.data bash -c
    mkfs.btrfs -f $raw 2>/dev/null >/dev/null
    dir=zstd${compress_level}.dir
    mkdir -p $dir
    if [ $compress_level -eq 0 ] ; then
        mount $raw $dir
    else
        mount -o compress=zstd:${compress_level} $raw $dir
    fi
    echo Compression Level ${compress_level}
    time bash -c \
         "dd if=blah of=$dir/blah ; sync $dir"
done

Results

Compression Level 0
4194304+0 records in
4194304+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 13.3657 s, 161 MB/s

real    0m13.386s
user    0m2.891s
sys     0m9.486s
Compression Level 1
4194304+0 records in
4194304+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 14.8094 s, 145 MB/s

real    0m14.913s
user    0m3.313s
sys     0m10.380s
Compression Level 3
4194304+0 records in
4194304+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 15.1291 s, 142 MB/s

real    0m15.259s
user    0m3.261s
sys     0m10.720s
Compression Level 9
4194304+0 records in
4194304+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 15.4792 s, 139 MB/s

real    0m15.499s
user    0m3.442s
sys     0m10.913s

/dev/zero cpu copy test

tl;dr;

Easily compressed data is a lot more expensive at higher levels. zstd:1 added 16%, zstd:9 added 82%

Test Steps

Same as above, but swap /dev/urandom for /dev/zero

Results

Compression Level 0
4194304+0 records in
4194304+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 14.1791 s, 151 MB/s

real    0m14.196s
user    0m2.971s
sys     0m9.909s
Compression Level 1
4194304+0 records in
4194304+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 16.4391 s, 131 MB/s

real    0m16.536s
user    0m3.403s
sys     0m11.511s
Compression Level 3
4194304+0 records in
4194304+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 14.9807 s, 143 MB/s

real    0m15.094s
user    0m3.398s
sys     0m10.833s
Compression Level 9
4194304+0 records in
4194304+0 records out
2147483648 bytes (2.1 GB, 2.0 GiB) copied, 18.0451 s, 119 MB/s

real    0m18.066s
user    0m3.758s
sys     0m12.173s