To maximize the endurance of a LiveOS image, several elements of the LiveOS design deserve consideration.
Avoid persistent overlay consumption
- Use a persistent home.img file system to mount on the /home folder (discussed on main page, LiveOS image#Home filesystem).
- This is a very effective method to avoid consumption and will also make the user files more available for sharing and backup purposes.
- Mount more folders onto temporary, in-memory filesystems (like /var/tmp, & /tmp are now).
- /var/log/audit (holds the SELinux audit.log that records a great many file accesses.)
- /var/spool/abrt (holds often large, error reports and core dumps). Users could be advised to act on any abrt reports in the current session or copy the reports to other permanent storage, such as in /home/ or external storage.
- there may be other good candidates...
Overlay recovery
- On Fedora 24+, if the overlay storage space on a LiveOS device is completely filled, Device-mapper will report 'Overflow' status:
# dmsetup status live-base: 0 13635584 linear live-rw: 0 13635584 snapshot Overflow
- While the system continues to function, the root filesystem has been remounted read-only. You will probably see errors due to this restriction.
- Once the system is shut down, possibly by a hard reset, you may attempt to recover the filesystem with the procedure below, though success is not certain.
First, determine the size of the overlay file:
# losetup /dev/loop0 /run/media/user/devicename/LiveOS/overlay-devicename-discUUID -r # blockdev -q --getsz /dev/loop0 # losetup -d /dev/loop0
Next, enlarge the overlay:
# dd if=/dev/zero of=/path/to/overlay-file seek=overlay_size count=size_increase conv=fsync
- where overlay_size and size_increase are given in 512 byte units.
Then, determine the size of the root filesystem:
# mount /run/media/user/devicename/LiveOS/squashfs.img /mnt/some_mountpoint -r # losetup /dev/loop1 /mnt/some_mountpoint/LiveOS/rootfs.img -r # blockdev -q --getsz /dev/loop1
And register the LiveOS image snapshot with Device-mapper:
# losetup /dev/loop2 /run/media/user/devicename/LiveOS/overlay-devicename-discUUID # dmsetup create devicename --table "0 13635584 snapshot 7:1 7:2 PO 8"
- devicename in the last line may be any string.
- loop1 and loop2 (and the corresponding 7:1 7:2) may be any free loop devices; just substitute the appropriate node numbers.
Check that the the virtual filesystem has been configured:
# dmsetup status
Finally, try to repair any corruption in the filesystem with the following command:
# e2fsck -f -y /dev/mapper/devicename
- where devicename is the string you used in the dmsetup create command.
Run a second check,
# e2fsck -p /dev/mapper/devicename
- to verify if the filesystem could be repaired.
Once you have a working filesystem, you may want to merge your overlay into the original filesystem as described below to recover unused storage space.
- On builds before Fedora 24, if one exhausts the limited storage capacity of a LiveOS overlay, Device-mapper will mark the filesystem as 'Invalid', as shown by the dmsetup status command executed in Terminal or a console (if you haven't crashed):
# dmsetup status live-osimg-min: 0 8388608 snapshot 2464/2464 24 live-rw: 0 8388608 snapshot Invalid
- The invalid flag is 00 at byte 5 of the overlay.
Switch the invalid flag to 01 with the following command line:
# echo $'\x01' | dd of=/path/to/overlay-file bs=1 count=1 seek=4 conv=notrunc
Then, proceed with the recovery instructions, above, as given for the overflow condition.
Merge overlay into new image
The editliveos Python script in the livecd-tools package will merge overlays into a new base image. The BASH script below may help one to understand what steps are involved.
A new root filesystem image can be constructed by using some Device-mapper tools.
If one has sufficient free disk space available, typically on an attached hard drive, one can copy (and uncompress) the current filesystem to a working folder. (This may require, for example, over 4.0 GiB of free space for Fedora 24 Workstation Live.)
Device-mapper's mirror target allows one to create a 'mirror' image of the LiveOS root filesystem (the snapshot based on the SquashFS and overlay device). The mirror image can be then be recompressed, and if one has an additional 1.5 GiB or more of free space (more depending on how much software or other root filesystem files have been added) on your underlying device filesystem (the USB/SD card filesystem), one can delete the old SquashFS image and replace it with the recompressed version.
The snapshot overlay can then be reset before rebooting the LiveOS device.
The following Bash script demonstrates the process, where the $1 parameter is a mount point directory for a disk partition with sufficient storage capacity (a LiveOS folder will be created there for holding the mirrored filesystem), and $2 is the LiveOS device node name (such as, /run/initramfs/livedev, for the currently booted LiveOS image, or /dev/sdc1, for example, for an attached LiveOS device). Options xtrace and verbose have been set to aid debugging.
#!/bin/bash -xv # # Merge a LiveOS snapshot overlay into a new root filesystem, recompress it # into a SquashFS image (if the source was so packaged), replace the source # with the refresh, and reset the overlay. # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation; version 2 of the License. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU Library General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, write to the Free Software # Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307, USA. oldLC=$LC_NUMERIC LC_NUMERIC="en_US.UTF-8" export PATH=/usr/sbin:$PATH export \ PS4='+(${SHLVL}:${LINENO}:${BASH_SOURCE}:${EUID}): ${FUNCNAME[0]:+${FUNCNAME[0]}(): }' set -o errexit set -o pipefail t0=$(date +%s) cleanup () { umount -l $SRCMNT rmdir $SRCMNT dt=$(($(date +%s)-t0)) h=$((dt/3600)) m=$((dt%3600/60)) s=$((dt%60)) printf '\nTotal time elapsed: %02d:%02d:%02d hh:mm:ss\n' $h $m $s [[ $1 == 0 ]] && exit 0 || exit 1 } if [[ ! -f /usr/sbin/mksquashfs ]]; then echo "Please install squashfs-tools (yum install squashfs-tools)"; exit fi TMPDIR=$1 [[ ! -d $TMPDIR/LiveOS ]] && mkdir $TMPDIR/LiveOS 2>/dev/null # Mount the source device and SquashFS. mkdir -p /run/media #Doesn't exist in fedora 20 livecd SRCMNT=$(mktemp -d /run/media/XXXXXX) mount $2 $SRCMNT if [[ -e $SRCMNT/LiveOS/squashfs.img ]]; then SQUASHMNT=$(mktemp -d /run/media/XXXXXX) mount $SRCMNT/LiveOS/squashfs.img $SQUASHMNT --read-only ORIGFS=$SQUASHMNT/LiveOS/ext3fs.img [[ ! -f $ORIGFS ]] && ORIGFS=$SQUASHMNT/LiveOS/rootfs.img else ORIGFS=$SRCMNT/LiveOS/ext3fs.img [[ ! -f $ORIGFS ]] && ORIGFS=$SRCMNT/LiveOS/rootfs.img fi OVERLAY=/LiveOS/$(basename $SRCMNT/LiveOS/overlay-*) if [[ -f $SRCMNT$OVERLAY ]]; then OVDEV=$(losetup -f --show $SRCMNT$OVERLAY --read-only) else printf '\n No persistent overlay for the filesystem was found. Exiting...\n' cleanup 1 fi # >= Fedora 17 LiveOSMNT=/run/initramfs/live # < Fedora 17 [[ -d /mnt/live/LiveOS ]] && LiveOSMNT=/mnt/live # If we are processing a booted LiveOS device's root filesystem, then if [[ $(mountpoint -x $2) == $(mountpoint -dq $LiveOSMNT) ]]; then # Remove live-osimg-min; free its loop devices. if [[ -e $SRCMNT/LiveOS/osmin.img ]]; then dmsetup remove live-osimg-min || : losetup -d /dev/loop1 || : losetup -d /dev/loop0 || : fi else LiveOSMNT='' fi # Remove expiring osmin.img rm $SRCMNT/LiveOS/osmin.img || : # Prepare temporary devices for the Device-mapper mirror target. FSTYPE=$(blkid -s TYPE -o value $ORIGFS || :) FSLABEL=$(blkid -s LABEL -o value $ORIGFS || :) stinfo=($(stat -c '%B %s %o' $ORIGFS || :)) BLOCKSZ=${stinfo[0]} BLOCKS=$((${stinfo[1]}/$BLOCKSZ)) IOBLKSZ=${stinfo[2]} NEWFS=$TMPDIR/LiveOS/rootfs.img dd if=/dev/null of=$NEWFS count=1 bs=$BLOCKSZ seek=$BLOCKS mkfs.$FSTYPE -F -L $FSLABEL -m 1 -b $IOBLKSZ $NEWFS tune2fs -c0 -i0 -Odir_index -ouser_xattr,acl $NEWFS NEWFSDEV=$(losetup -f --show $NEWFS) roORIGFSDEV=$(losetup -f --show $ORIGFS --read-only) table="0 $BLOCKS snapshot $roORIGFSDEV $OVDEV P 8" dmsetup create ori --readonly --table "$table" # Invoke mirror target device. table="0 $BLOCKS mirror core 2 32 sync 2 /dev/mapper/ori 0 $NEWFSDEV 0" dmsetup create mir --table "$table" set +xv # Wait for mirror completion. while state=$(dmsetup status mir) state=${state#*mirror 2 * * } alloc=${state%/*} size=${state#*/} size=${size%% *} [[ $alloc != $size ]]; do percent=$(dc <<< "8k 10 0.05 100 $alloc $size /*+*p") percent=$(dc <<< "1k $percent 10 /p") printf '\r Mirroring %5.1f %% complete. ' $percent printf '\b|' sleep 0.5 printf '\b/' sleep 0.5 printf '\b-' sleep 0.5 printf '\b\' sleep 0.5 done printf '\r Mirroring 100 %% complete. \n' set -xv # Check the new filesystem. e2fsck -f -y $NEWFS || e2fsck -f -y $NEWFS # Clean up. dmsetup remove mir ori sync dt=$(($(date +%s)-t0)) h=$((dt/3600)) m=$((dt%3600/60)) s=$((dt%60)) printf '\nTime elapsed: %02d:%02d:%02d hh:mm:ss\n' $h $m $s # Replace SquashFS or uncompressed root filesystem image. if [[ -e $SRCMNT/LiveOS/squashfs.img ]]; then umount -l $SQUASHMNT rm $SRCMNT/LiveOS/squashfs.img mksquashfs $TMPDIR/LiveOS $SRCMNT/LiveOS/squashfs.img -comp xz -keep-as-directory else if [[ -x /usr/bin/rsync ]]; then rsync --inplace --progress $NEWFS $SRCMNT/LiveOS else cp $NEWFS $SRCMNT/LiveOS fi fi # Clean up. losetup -d $OVDEV $NEWFSDEV $roORIGFSDEV OVDEV=$(losetup -f --show $SRCMNT$OVERLAY) # Reset overlay. dd if=/dev/zero of=$OVDEV bs=64k count=1 conv=notrunc,fsync # If we are processing a booted LiveOS device's root filesystem, then if [[ -n $LiveOSMNT ]]; then # Try to unmount the TMPDIR. mountpoint $TMPDIR && umount -l $TMPDIR || : shutdown -r +1 'The system will reboot in 1 minute.' else sleep 1 losetup -d $OVDEV fi cleanup 0
Note: If the above script is invoked on a currently booted LiveOS root filesystem, the system will be scheduled for a reboot one minute after the script completes.
The Device-mapper snapshot-merge target also allows one to merge changes in a persistent snapshot into the original filesystem (See notes from Andrew Gilmore).
Merge overlay with snapshot-merge |
---|
#!/bin/bash t0=$(date +%s) # Prepare working directory. TMPDIR=$1 mkdir $TMPDIR/LiveOS 2>/dev/null # Mount source device, SquashFS, if needed. SRCMNT=$(mktemp -d /media/XXXXXX) mount $2 $SRCMNT if [[ -e $SRCMNT/LiveOS/squashfs.img ]]; then SQUASHMNT=$(mktemp -d /media/XXXXXX) mount $SRCMNT/LiveOS/squashfs.img $SQUASHMNT --read-only ORIGFS=$SQUASHMNT/LiveOS/ext3fs.img else ORIGFS=$SRCMNT/LiveOS/ext3fs.img fi NEWFS=$TMPDIR/LiveOS/ext3fs.img printf '\nCopying original root filesystem.\n' if [[ -x /usr/bin/rsync ]]; then rsync --inplace --progress $ORIGFS $NEWFS else cp $ORIGFS $NEWFS fi # Prepare temporary devices for Device-mapper snapshot-merge. BLOCKS=$(stat -c '%b' $ORIGFS || :) NEWFSDEV=$(losetup -f --show $NEWFS) OVDEV=$(losetup -f --show $SRCMNT/LiveOS/overlay-*) # Invoke snapshot-merge target device. dmsetup create merge <<< "0 $BLOCKS snapshot-merge $NEWFSDEV $OVDEV P 8" # Wait for merge completion. while state=$(dmsetup status merge) state=${state#*snapshot-merge } meta=${state#* } alloc=${state%/*} [[ $alloc != $meta ]]; do printf '/' sleep 0.5 printf '\b-' sleep 0.5 printf '\b\' sleep 0.5 printf '\b|' sleep 0.5 done printf '\n' # Clean up from merge. dmsetup remove merge sync dt=$(($(date +%s)-t0)) h=$((dt/3600)) m=$((dt%3600/60)) s=$((dt%60)) printf '\nTime elapsed: %02d:%02d:%02d hh:mm:ss\n' $h $m $s # Replace original image if [[ -e $SRCMNT/LiveOS/squashfs.img ]]; then umount $SQUASHMNT rmdir $SQUASHMNT rm $SRCMNT/LiveOS/squashfs.img mksquashfs $TMPDIR/LiveOS $SRCMNT/LiveOS/squashfs.img -comp xz -keep-as-directory else if [[ -x /usr/bin/rsync ]]; then rsync --inplace --progress $NEWFS $SRCMNT/LiveOS else cp $NEWFS $SRCMNT/LiveOS fi fi # Reset overlay. dd if=/dev/zero of=$OVDEV bs=64k count=1 conv=notrunc,fsync # Clean up. sleep 2 losetup -d $OVDEV $NEWFSDEV sleep 2 umount $SRCMNT rmdir $SRCMNT LC_NUMERIC=$oldLC dt=$(($(date +%s)-t0)) h=$((dt/3600)) m=$((dt%3600/60)) s=$((dt%60)) printf '\nTotal time elapsed: %02d:%02d:%02d hh:mm:ss\n' $h $m $s |
The mirror method copies and merges in one pass, while the snapshot-merge method copies the original filesystem first and then merges in the changes. I've found the mirror method to be about 15% faster.
editliveos is a Python script that performs overlay merging and, optionally, other Live OS editing. It is available in Version 25 of the livecd-tools
package.
Credits
- The method of mirroring the LiveOS was adapted from Douglas McClendon's ZyX-LiveInstaller.