From Fedora Project Wiki

(Avoid a warning message to prevent confusion.)
(Note this is being run as part of CI now)
 
(12 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{QA/Test_Case
{{QA/Test_Case
|description=Test that compression is reversible for squashfs-tools.
|description=Test that compression is reversible for {{package|squashfs-tools}}. The compression types tested will be those expected to work with the version of squashfs-tools you have installed and the kernel you are running. These tests are used to make sure squashfs-tools will work as expected for live images.
|actions=
# You need to have squashfs-tools and rpmdevtools installed.
# Run the provided script as root and note any failure messages. The script needs to be run as root so that squashfs images can be mounted and to prevent warnings about setting selinux extended attributes when running unsquashfs.


The compression types tested will be those expected to work with the version of squashfs-tools you have installed and the kernel you are running.
This test case should be being run as part of continuous integration testing. That currently will only will cover x86_64 and will only get checked against one kernel version. So there can be value in running this manually still.
 
The script will create a directory in the current working directory named "test-squashfs". When run, the script will check if "test-squashf/data" is already created and if so won't recreate the test data.


A test script is available to provide a consistent test result.  The test script needs to be run as the ''root'' user so that squashfs images can be mounted and to prevent warnings about setting selinux extended attributes when running unsquashfs.  The test script will also create a directory in the current working directory named {{filename|test-squashfs}}.  When run, the script will check if {{filename|test-squashfs/data}} is already created and if so won't recreate the test data.
|setup=
# Install the required {{package|squashfs-tools}} and {{package|rpmdevtools}} packages: <pre>yum install squashfs-tools rpmdevtools</pre>
|actions=
# Download the test script [https://fedoraproject.org/w/uploads/2/25/Squashfs-compression-test.sh Squashfs-compression-test.sh]
# As root, execute the test script using the command: {{command|bash Squashfs-compression-test.sh}}.  Note any failure messages.
|results=
<ol>
<li>The test script should exit cleanly and not abort early because it was unable to create some file or free disc space running out. There should be no messages with the string <code>failed</code> in them. Typically you might run this using script and then grep <code>typescript</code> for <code>failed</code>.  Successful test output is included below:
<pre><nowiki>
<pre><nowiki>
#!/bin/bash
Building data directory.
 
Building squashfs image using gzip compression.
# We need rpmdev-vercmp
Parallel mksquashfs: Using 2 processors
rpm -q rpmdevtools > /dev/null 2>&1 || (echo 'rpmdev-vercmp from rpmdevtools is needed to run this script, aborting.'; exit 1)
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
 
[=============================================================|] 2316/2316 100%
# Define block sizes
Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
blocks=(4K 1M)
compressed data, compressed metadata, compressed fragments, compressed xattrs
 
duplicates are removed
# Define fill files
Filesystem size 107770.89 Kbytes (105.25 Mbytes)
fill=(/dev/zero /dev/urandom)
38.79% of uncompressed filesystem size (277839.40 Kbytes)
 
Inode table size 2185 bytes (2.13 Kbytes)
# Define number of iterations
11.16% of uncompressed inode table size (19586 bytes)
iter=5
Directory table size 1442 bytes (1.41 Kbytes)
 
21.57% of uncompressed directory table size (6684 bytes)
# Define fragment sizes
Xattr table size 54 bytes (0.05 Kbytes)
frags=(0 1 2047 4095)
100.00% of uncompressed xattr table size (54 bytes)
 
Number of duplicate files found 42
# Define test directory
Number of inodes 189
testdir=./test-squashfs
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
root (0)
Number of gids 1
root (0)
Testing unmounted extract using gzip compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write


# Define mount point
[=============================================================-] 2316/2316 100%
mp=${testdir}/mnt
created 188 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Testing mounted image using gzip compression.
Building squashfs image using lzo compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================/] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, lzo compressed, data block size 131072
compressed data, compressed metadata, compressed fragments, compressed xattrs
duplicates are removed
Filesystem size 107764.62 Kbytes (105.24 Mbytes)
38.79% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 2553 bytes (2.49 Kbytes)
13.03% of uncompressed inode table size (19586 bytes)
Directory table size 1732 bytes (1.69 Kbytes)
25.91% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
root (0)
Number of gids 1
root (0)
Testing unmounted extract using lzo compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write


# Define data directory
[=============================================================/] 2316/2316 100%
datadir=${testdir}/data
created 188 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Testing mounted image using lzo compression.
Building squashfs image using lzma compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================|] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, lzma compressed, data block size 131072
compressed data, compressed metadata, compressed fragments, compressed xattrs
duplicates are removed
Filesystem size 107750.75 Kbytes (105.23 Mbytes)
38.78% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 1747 bytes (1.71 Kbytes)
8.92% of uncompressed inode table size (19586 bytes)
Directory table size 1209 bytes (1.18 Kbytes)
18.09% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
root (0)
Number of gids 1
root (0)
Testing unmounted extract using lzma compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write


# Check for squashfs-tools version and set compression types to test
[=============================================================-] 2316/2316 100%
sqfsver=`rpm -q --qf '%{EVR}' squashfs-tools`
created 188 files
if rpmdev-vercmp 4.1 ${sqfsver} > /dev/null; [ $? == '11' ]; then
created 1 directories
  ucomp=('gzip')
created 0 symlinks
elif rpmdev-vercmp 4.2 ${sqfsver} > /dev/null; [ $? == '11' ]; then
created 0 devices
  ucomp=(gzip lzo lzma)
created 0 fifos
else
Building squashfs image using xz compression.
  ucomp=(gzip lzo lzma xz)
Parallel mksquashfs: Using 2 processors
fi
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
 
[=============================================================/] 2316/2316 100%
# Check for kernel verion and set mount test compression types
Exportable Squashfs 4.0 filesystem, xz compressed, data block size 131072
kernel=`uname -r`
compressed data, compressed metadata, compressed fragments, compressed xattrs
if rpmdev-vercmp 2.6.36 ${kernel} > /dev/null; [ $? == '11' ]; then
duplicates are removed
  mcomp=('gzip')
Filesystem size 107751.25 Kbytes (105.23 Mbytes)
elif rpmdev-vercmp 2.6.38 ${kernel} > /dev/null; [ $? == '11' ]; then
38.78% of uncompressed filesystem size (277839.40 Kbytes)
  mcomp=(gzip lzo)
Inode table size 1874 bytes (1.83 Kbytes)
else
9.57% of uncompressed inode table size (19586 bytes)
  mcomp=(gzip lzo xz)
Directory table size 1262 bytes (1.23 Kbytes)
fi
18.88% of uncompressed directory table size (6684 bytes)
 
Xattr table size 54 bytes (0.05 Kbytes)
# Check for uid 0 and print a warning if not
100.00% of uncompressed xattr table size (54 bytes)
[ ${UID} -ne 0 ] && echo 'Mount tests will not be performed when not running as root'
Number of duplicate files found 42
 
Number of inodes 189
# Check if test directory exists and make if not
Number of files 188
[ -d ${testdir} ] || mkdir ${testdir}
Number of fragments 6
[ -d ${testdir} ] || (echo "Unable to make '${testdir}', aborting."; exit 1)
Number of symbolic links  0
 
Number of device nodes 0
# Check if mount point directory exists and make if not
Number of fifo nodes 0
[ -d ${mp} ] || mkdir ${mp}
Number of socket nodes 0
[ -d ${mp} ] || (echo "Unable to make '${mp}', aborting."; exit 1)
Number of directories 1
 
Number of ids (unique uids + gids) 1
# Check if data directory exists and make if not
Number of uids 1
if [ -d ${datadir} ]; then
root (0)
  echo "Using existing data directory."
Number of gids 1
else
root (0)
  echo "Building data directory."
Testing unmounted extract using xz compression.
  mkdir ${datadir}
Parallel unsquashfs: Using 2 processors
  [ -d ${datadir} ] || (echo "Unable to make '${datadir}', aborting."; exit 1)
188 inodes (2316 blocks) to write
  for size in ${frags[*]}; do
    for file in ${fill[*]}; do
      dd if=${file} of=${datadir}/frag-`basename ${file}`-${size} bs=1 count=${size} > /dev/null 2>&1
    done
  done
  for size in ${blocks[*]}; do
    for ((count=1;${count}<=${iter};count++)); do
      for file in ${fill[*]}; do
        dd if=${file} of=${datadir}/file-`basename ${file}`-${size}-${count} bs=${size} count=${count} > /dev/null 2>&1
      done
    done
  done
  for size1 in ${frags[*]}; do
    for file1 in ${fill[*]}; do
      for size2 in ${blocks[*]}; do
        for ((count=1;${count}<=${iter};count++)); do
          for file2 in ${fill[*]}; do
            cat ${datadir}/file-`basename ${file2}`-${size2}-${count} ${datadir}/frag-`basename ${file1}`-${size1} > ${datadir}/combined-`basename ${file2}`-${size2}-${count}-`basename ${file1}`-${size1}
          done
        done
      done
    done
  done
fi
 
# Run unmounted tests
for comp in ${ucomp[*]}; do
  echo "Building squashfs image using ${comp} compression."
  if [ "${comp}" == gzip ]; then
    mksquashfs ${datadir} ${testdir}/sq.img || (echo "mksquashfs failed for ${comp} compression."; continue)
  else
    mksquashfs ${datadir} ${testdir}/sq.img -comp ${comp} || (echo "mksquashfs failed for ${comp} compression."; continue)
  fi
  echo "Testing unmounted extract using ${comp} compression."
  unsquashfs -d ${testdir}/sq ${testdir}/sq.img  || echo "unsquashfs failed for ${comp} compression."
  diff -r -q ${testdir}/sq ${datadir} || echo "Extract test failed for ${comp} compression."
  rm -rf ${testdir}/sq
  if [ ${UID} == 0 ]; then
    for kern in ${mcomp[*]}; do
      if [ ${kern} == ${comp} ]; then
        echo "Testing mounted image using ${comp} compression."
        mount -r -o loop ${testdir}/sq.img ${mp} || echo "Mount failed.";
        diff -r -q ${mp} ${datadir} || echo "Mounted test failed for ${comp} compression."
        umount ${mp}
        break
      fi
    done
  fi
  rm -f ${testdir}/sq ${testdir}/sq.img
done


[=============================================================/] 2316/2316 100%
created 188 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
</nowiki></pre>
</nowiki></pre>
 
</li></ol>
|results=
# The test should not abort early because it was unable to create some file or free disc space running out. There should be no messages with the string "failed" in them. Typically you might run this using script and then grep "typescript" for "failed".
}}
}}
[[Category:Critical_path_test_cases]]
[[Category:Critical_path_test_cases]]
[[Category:Package_squashfs-tools_test_cases]]

Latest revision as of 11:34, 23 May 2019

Description

Test that compression is reversible for Package-x-generic-16.pngsquashfs-tools. The compression types tested will be those expected to work with the version of squashfs-tools you have installed and the kernel you are running. These tests are used to make sure squashfs-tools will work as expected for live images.

This test case should be being run as part of continuous integration testing. That currently will only will cover x86_64 and will only get checked against one kernel version. So there can be value in running this manually still.

A test script is available to provide a consistent test result. The test script needs to be run as the root user so that squashfs images can be mounted and to prevent warnings about setting selinux extended attributes when running unsquashfs. The test script will also create a directory in the current working directory named test-squashfs. When run, the script will check if test-squashfs/data is already created and if so won't recreate the test data.

Setup

  1. Install the required Package-x-generic-16.pngsquashfs-tools and Package-x-generic-16.pngrpmdevtools packages: 
    yum install squashfs-tools rpmdevtools

How to test

  1. Download the test script Squashfs-compression-test.sh
  2. As root, execute the test script using the command: bash Squashfs-compression-test.sh. Note any failure messages.

Expected Results

  1. The test script should exit cleanly and not abort early because it was unable to create some file or free disc space running out. There should be no messages with the string failed in them. Typically you might run this using script and then grep typescript for failed. Successful test output is included below: 
    Building data directory.
Building squashfs image using gzip compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================|] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
	compressed data, compressed metadata, compressed fragments, compressed xattrs
	duplicates are removed
Filesystem size 107770.89 Kbytes (105.25 Mbytes)
	38.79% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 2185 bytes (2.13 Kbytes)
	11.16% of uncompressed inode table size (19586 bytes)
Directory table size 1442 bytes (1.41 Kbytes)
	21.57% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
	100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
	root (0)
Number of gids 1
	root (0)
Testing unmounted extract using gzip compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write

[=============================================================-] 2316/2316 100%
created 188 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Testing mounted image using gzip compression.
Building squashfs image using lzo compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================/] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, lzo compressed, data block size 131072
	compressed data, compressed metadata, compressed fragments, compressed xattrs
	duplicates are removed
Filesystem size 107764.62 Kbytes (105.24 Mbytes)
	38.79% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 2553 bytes (2.49 Kbytes)
	13.03% of uncompressed inode table size (19586 bytes)
Directory table size 1732 bytes (1.69 Kbytes)
	25.91% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
	100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
	root (0)
Number of gids 1
	root (0)
Testing unmounted extract using lzo compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write

[=============================================================/] 2316/2316 100%
created 188 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Testing mounted image using lzo compression.
Building squashfs image using lzma compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================|] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, lzma compressed, data block size 131072
	compressed data, compressed metadata, compressed fragments, compressed xattrs
	duplicates are removed
Filesystem size 107750.75 Kbytes (105.23 Mbytes)
	38.78% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 1747 bytes (1.71 Kbytes)
	8.92% of uncompressed inode table size (19586 bytes)
Directory table size 1209 bytes (1.18 Kbytes)
	18.09% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
	100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
	root (0)
Number of gids 1
	root (0)
Testing unmounted extract using lzma compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write

[=============================================================-] 2316/2316 100%
created 188 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Building squashfs image using xz compression.
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on ./test-squashfs/sq.img, block size 131072.
[=============================================================/] 2316/2316 100%
Exportable Squashfs 4.0 filesystem, xz compressed, data block size 131072
	compressed data, compressed metadata, compressed fragments, compressed xattrs
	duplicates are removed
Filesystem size 107751.25 Kbytes (105.23 Mbytes)
	38.78% of uncompressed filesystem size (277839.40 Kbytes)
Inode table size 1874 bytes (1.83 Kbytes)
	9.57% of uncompressed inode table size (19586 bytes)
Directory table size 1262 bytes (1.23 Kbytes)
	18.88% of uncompressed directory table size (6684 bytes)
Xattr table size 54 bytes (0.05 Kbytes)
	100.00% of uncompressed xattr table size (54 bytes)
Number of duplicate files found 42
Number of inodes 189
Number of files 188
Number of fragments 6
Number of symbolic links  0
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 1
Number of ids (unique uids + gids) 1
Number of uids 1
	root (0)
Number of gids 1
	root (0)
Testing unmounted extract using xz compression.
Parallel unsquashfs: Using 2 processors
188 inodes (2316 blocks) to write

[=============================================================/] 2316/2316 100%
created 188 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
Retrieved from "https://fedoraproject.org/w/index.php?title=QA:Testcase_squashfs-tools_compression&oldid=544007"