Cloud image generation short cuts for noobs

Cloud image generation short cuts for noobs

Introduction

References

• http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?creating-loopback-s3-linux.html
• http://aws.amazon.com/developertools/351?_encoding=UTF8&jiveRedirect=1 The tools download page.
• http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?creating-an-ami.html
• http://docs.amazonwebservices.com/AWSEC2/2011-05-15/CommandLineReference/
• http://www.linuxplanet.com/linuxplanet/tutorials/6546/1
• http://alestic.com/2010/01/ec2-ebs-boot-ubuntu
• http://cssoss.wordpress.com/2010/05/10/eucalyptus-beginner%E2%80%99s-guide-%E2%80%93-uec-edition-chapter-4-%E2%80%93-image%C2%A0management/
• http://rwmj.wordpress.com/2011/05/19/rhel-6-1-is-out-new-virtualization-guide-covers-libguestfs/
• https://help.ubuntu.com/community/UEC/CDInstall
• https://help.ubuntu.com/community/UEC/Images
• http://cloud.ubuntu.com/docs/uec-book/
• http://cloud.ubuntu.com/docs/
• http://home.comcast.net/~fbui/bandwidth.html
• http://www.osc.edu/~troy/lperfex/
• [Home page]

Preping the environment

Getting the credentials

Eucalyptus credentials

See: https://help.ubuntu.com/community/UEC/CDInstall#STEP%205:%20Obtain%20Credentials

Creating an image

1. Get the OS
2. create loop back image
3. configure image.
4. bundle image.
5. Install the bundle on UEC.
6. Test the Ubuntu i386 bundle on AWS.
7. Test the bundle on AWS

Installing the tools

Installing the EC2 tools

1. get the tools zip file: EC2 AMI Tools
   • The AMI tools uses ruby: yum install ruby
2. cd /opt
3. unzip ec2-ami-tools.zip
4. export JAVA_HOME=/usr
5. export EC2_AMITOOL_HOME=/opt/ec2-ami-tools-1.3-66634
6. export PATH=$PATH:${EC2_AMITOOL_HOME:-EC2_HOME}/bin

For the API tools

1. export EC2_HOME=/opt/ec2-api-tools-1.4.3.0

• Use the PATH set above. export PATH=$PATH:$EC2_HOME/bin

Creating a Fedora image

1. dd if=/dev/zero of=fedora.fs bs=1M count=2048
2. mke2fs -F -j fedora.fs
3. mkdir /mnt/fedora
4. mount -o loop fedora.fs /mnt/fedora
5. mkdir /mnt/fedora/dev
6. /sbin/MAKEDEV -d /mnt/fedora/dev -x console
7. /sbin/MAKEDEV -d /mnt/fedora/dev -x null
8. /sbin/MAKEDEV -d /mnt/fedora/dev -x zero
9. mkdir /mnt/fedora/etc
10. vi /mnt/fedora/etc/fstab
    • See fstab content below.
11. cat /etc/yum.conf /etc/yum.repos.d/fedora.repo >> /mnt/fedora/etc/yum.conf
12. vi /mnt/fedora/etc/yum.conf
    • See yum.conf content below.
13. mkdir /mnt/fedora/proc
14. mount -t proc none /mnt/fedora/proc
15. yum -c /mnt/fedora/etc/yum.conf --installroot=/mnt/fedora -y groupinstall Base
16. vi /mnt/fedora/etc/sysconfig/network-scripts/ifcfg-eth0
    • See blow
17. echo "NETWORKING=yes" > /mnt/fedora/etc/sysconfig/network
18. Update /mnt/fedora/etc/fstab
    • See below
19. chroot /mnt/ec2-fs /bin/sh
20. chkconfig --level 345 my-service on
21. exit
22. umount /mnt/fedora/proc/
23. umount -d /mnt/fedora
24. ec2-bundle-image -i /disk2/fedora.fs -k ~/.euca/mykey.priv -c ~/.euca/euca2-ME-CODE-cert.pem -u 123456789012 -r x86_64 -d /disk2/product

fstab

/dev/sda1  /         ext3    defaults        1 1
none       /dev/pts  devpts  gid=5,mode=620  0 0
none       /dev/shm  tmpfs   defaults        0 0
none       /proc     proc    defaults        0 0
none       /sys      sysfs   defaults        0 0

# This is for c1.small and m1.medium
#  For others please see:
http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/index.html?instance-storage-concepts.html
/dev/sda2  /mnt      ext3    defaults        0 0
/dev/sda3  swap      swap    defaults        0 0

yum.conf

[main]
cachedir=/mnt/fedora/var/cache/yum/$basearch/$releasever
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
exactarch=1
obsoletes=1
plugins=1
installonly_limit=3
color=never
exclude=*-debuginfo
gpgcheck=0
reposdir=/dev/null

#  This is the default, if you make this bigger yum won't see if the metadata # is newer on the remote and so you'll "gain" the bandwidth of not having to # download the new metadata and "pay" for it by yum not having correct # information.
#  It is esp. important, to have correct metadata, for distributions like # Fedora which don't keep old packages around. If you don't like this checking # interupting your command line usage, it's much better to have something # manually check the metadata once an hour (yum-updatesd will do this).
# metadata_expire=90m

# PUT YOUR REPOS HERE OR IN separate files named file.repo # in /etc/yum.repos.d

[fedora]
name=Fedora $releasever - $basearch
failovermethod=priority
#mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever&arch=$basearch
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=fedora-14&arch=$basearch
enabled=1
metadata_expire=7d
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$basearch

[fedora-debuginfo]
name=Fedora $releasever - $basearch - Debug failovermethod=priority #baseurl=http://download.fedoraproject.org/pub/fedora/linux/releases/$releasever/Everything/$basearch/debug/
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=fedora-debug-$releasever&arch=$basearch
enabled=0
metadata_expire=7d
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$basearch

[fedora-source]
name=Fedora $releasever - Source
failovermethod=priority
[main]
cachedir=/mnt/fedora/var/cache/yum/$basearch/$releasever
keepcache=0
debuglevel=2
logfile=/var/log/yum.log
exactarch=1
obsoletes=1
plugins=1
installonly_limit=3
color=never
exclude=*-debuginfo
gpgcheck=0
reposdir=/dev/null

#  This is the default, if you make this bigger yum won't see if the metadata # is newer on the remote and so you'll "gain" the bandwidth of not having to # download the new metadata and "pay" for it by yum not having correct # information.
#  It is esp. important, to have correct metadata, for distributions like # Fedora which don't keep old packages around. If you don't like this checking # interupting your command line usage, it's much better to have something # manually check the metadata once an hour (yum-updatesd will do this).
# metadata_expire=90m

# PUT YOUR REPOS HERE OR IN separate files named file.repo # in /etc/yum.repos.d

[fedora]
name=Fedora $releasever - $basearch
failovermethod=priority
#mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=fedora-$releasever&arch=$basearch
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=fedora-14&arch=$basearch
enabled=1
metadata_expire=7d
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$basearch

[fedora-debuginfo]
name=Fedora $releasever - $basearch - Debug failovermethod=priority #baseurl=http://download.fedoraproject.org/pub/fedora/linux/releases/$releasever/Everything/$basearch/debug/
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=fedora-debug-$releasever&arch=$basearch
enabled=0
metadata_expire=7d
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$basearch

[fedora-source]
name=Fedora $releasever - Source
failovermethod=priority
#baseurl=http://download.fedoraproject.org/pub/fedora/linux/releases/$releasever/Everything/source/SRPMS/
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=fedora-source-$releasever&arch=$basearch
enabled=0
metadata_expire=7d
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$basearch
#baseurl=http://download.fedoraproject.org/pub/fedora/linux/releases/$releasever/Everything/source/SRPMS/
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=fedora-source-$releasever&arch=$basearch
enabled=0
metadata_expire=7d
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$basearch

ifcfg-eth0

DEVICE=eth0
BOOTPROTO=dhcp
ONBOOT=yes
TYPE=Ethernet
USERCTL=yes
PEERDNS=yes
IPV6INIT=no

Creating an Ubuntu image

• | Building an Ubuntu EMI: Ground Up

1. apt-get install python-vm-builder
2. vi image_def.txt

root 1000
/mnt/ephemeral 2000 /dev/sda2
swap 100 /dev/sda3
 

1. vmbuilder xen ubuntu --part ./image_def.txt
   • clear; vmbuilder qemu ubuntu -d /disk2/tmp/tut --verbose --part ./image_def.txt
2. mkdir /mnt/ubuntu
3. mount ubuntu-xen /mnt/ubuntu -o loop
4. chroot /mnt/ubuntu/ apt-get update
5. chroot /mnt/ubuntu/ apt-get install openssh-server
6. chroot /mnt/ubuntu/ passwd -d root
7. chroot /mnt/ubuntu vi /etc/rc.local
   • Add the code before the “exit 0“

depmod -a
modprobe acpiphp
 
# simple attempt to get the user ssh key using the meta-data service
mkdir -p /root/.ssh
echo >> /root/.ssh/authorized_keys
curl -m 10 -s http://169.254.169.254/latest/meta-data/public-keys/0/openssh-key | grep 'ssh-rsa' >> /root/.ssh/authorized_keys
echo "AUTHORIZED_KEYS:"
echo "************************"
cat /root/.ssh/authorized_keys
echo "************************"
 

1. chroot /mnt/ubuntu apt-get install curl
2. cp 2.6.28-11-generic /chroot/lib/modules -R
3. euca-bundle-image -i ubuntu-xen/root.img --kernel eki-CD7D185A --ramdisk eri-18301945 --prefix vmbuilder-test7
4. euca-upload-bundle -b imagestore-vmbuildertest6 -m /tmp/vmbuilder-test7.manifest.xml
5. euca-register imagestore-vmbuildertest6/vmbuilder-test7.manifest.xml
6. umount -l /mnt/ubuntu

Vanilla image creation

• | Creating an Eucalyptus image
• Make_QEMU_image_with_kickstart

Seems like EKI and ERI are something that is provide from outside the image. So it boot of of a kernel that is provided by the cloud vendor.

1. yum install qemu-img
2. qemu-img create -f qcow2 fedora.img 5G
3. qemu-kvm -m 256 -cdrom ../isos/fedora14.iso -drive file=fedora.img,if=scsi,index=0 -boot d -net nic -net user
   • Why use vnc?: -nographic -vnc :0

creating a RHEL server image on ubuntu

1. sudo su -
   • if the kvm is run as an ordinary user then there seems to be some timer hw issue in the guest.
2. qemu-img create -f qcow2 diskimage.img 5G
3. kvm -m 512 -cdrom rhel-server-6.1-i386-dvd.iso -drive file=diskimage.img,if=scsi,index=0 -boot d -net nic -net user