Moji
From FedoraProject
Contents |
Description
Moji is a simple build system in which the client grabs a package from the server, builds it locally using mock, and then uploads the results back to the original server. Moji stands for "mini koji" and it was written to support the Fedora-ARM pre-koji build effort (for example, F15-armv5tel).
Using Moji for the Fedora ARM Build Effort
Setting Up
To use a system as a Moji client for the ARM F15 armv5tel bringup:
- Install the Moji repo file:
rpm -Uvh http://scotland.proximity.on.ca/fedora-arm/moji-repo/rpm/moji-release-1-1.noarch.rpm - Install the Moji preconfigured client:
yum install -y moji-client - Start the Moji server:
service moji-client start - (Optional) Configure the Moji server to start at next boot:
chkconfig moji-client on
Please see the arm@lists.fedoraproject.org mailing list for more information.
- Alternative link: moji-client
Starting and Stopping the Moji Client
- To start the Moji client:
service moji-client start
- To keep the client active but in a state that does not accept new builds:
service moji-client lock
- To "unlock" a "locked" client (note, restarting the service accomplishes the same thing) :
service moji-client unlock
- To stop the Moji client immediately:
service moji-client stop
Viewing Current Status
- Go to the moji server status page (http://moji.proximity.on.ca/moji)
- The repo dir containing all mock output for stage0 (http://australia.proximity.on.ca/fedora-arm/f15v5.0/repo/)
Moji Source Code
The source code for Moji is in the Fedora ARM git repository at http://fedorahosted.org/arm/ and can be viewed at https://fedorahosted.org/arm/browser/moji/
Application Information
Server
python /path/to/moji.py server /path/to/source /path/to/working /path/to/finished /path/to/repo [password]
- "source" is the folder containing source RPM files
- the "source" folder can contain a file called "priority.txt" which is a simple plaintext file where each line is the filename of a source pkg that should be built first (ex. nled-2.52-7.fc15.src.rpm)
- "working" is a folder that contains any source RPM files currently being worked on by a builder
- "finished" is a folder containing source RPM files that have been built (failure or success)
- "repo" is the directory which contains the mock build result output of any "finished" builds
- the "repo" directory should also be publicly accessible (ex. via httpd)
- createrepo needs to also be run via some other means (ex. shell script via incrontab)
For the ARM F15 armv5tel startup, the Moji server is australia.proximity.on.ca.
Client
python /path/to/moji.py client <hostname or ip> <mock config name> [password]
- the mock option should be a config file name (ex fedora-15-arm)
- the actual mock file (ex. /etc/mock/fedora-15-arm.cfg) should point to at least 2 repos: one to the moji server repo dir and another to a pre-established "main-line" repo
Design/Protocol
- defaults to TCP port 6789
- non-threaded and does not fork per child
- does fork before any request that could take a while (ex. sending files or building packages)
- if a client disconnects, their build is considered failed and sent to another builder later on
- clients send dummy keep-alive ping messages thru its socket object
- more details to come soon...
Repo Management
$ vim repo.sh
- Note: Run the below script in a screen session
#!/bin/bash
export repo="/var/export/f15v5.0/repo"
export temp="/var/export/f15v5.0/temp"
while true
do
chek=`ls -lt ${repo}/*/*.[rx][pm][ml] | head -n 1 | tail -n 1 | grep -i '.*\.rpm$'`
if [ "${chek}" != "" ]
then
echo "Starting on [ `date` ]" | tee -a ~/repo.log
mkdir "${temp}" 2> /dev/null # make sure the temp repo directory exists
rm -fr ${temp}/* 2> /dev/null # clear the temp repo directory
ls "${repo}" | while read line
do
ln -s "${repo}/${line}" "${temp}/${line}"
done
rm -fr ${temp}/repodata # remove any temp repo data links
cp -fr ${repo}/repodata ${temp}/ # copy over the current repo data
createrepo --update -d "${temp}" # create/update the new repo data
cp -fr ${temp}/repodata ${repo}/ # copy back the new repo data
echo "Ending on [ `date` ]" | tee -a ~/repo.log
fi
sleep 5 # sleep loop or risk fire!
done
Base Package Set
priority.txt
- Use these package names only to get a list of full package file names (ENVRA)
- Each package entry must be on its own line in the text file
acl attr audit audit-libs basesystem bash binutils bison byacc bzip2 bzip2-libs ca-certificates ccache check chkconfig chrpath cloog cloog-ppl cluster cmake compat-readline5 coreutils coreutils-libs corosync cpio cpp cracklib cracklib-dicts crda cronie cryptsetup-luks curl cvs cyrus-sasl cyrus-sasl-lib dash db4 db4-utils dbus dbus-glib dbus-python dev dhcp diffutils dos2unix doxygen e2fsprogs ed elfutils elfutils-libelf elfutils-libs expat expect fakechroot fakeroot fedora-release file file-libs filesystem findutils firebird flex fontconfig freetype gamin gawk gc gcc gcc-c+ gcc-host gcc-libgcc gd gdb gdbm gettext git glib2 glibc glibc-common glibc-devel glibc-headers gmp gperf gpm grep gstreamer guile gzip hostname icu imake info initscripts iproute iputils java-1.5.0-gcj kbd kernel kernel-headers keyutils keyutils-libs krb5 krb5-libs less libacl libattr libblkid libcap libcom_err libcurl libffi libgcc libgomp libidn libmount libmpc libselinux libsepol libssh2 libstdc++ libstdc++-devel libutempter libuuid libxml2 linux-atm llvm lm_sensors lua lvm2 lzo m2crypto m4 make man-db mingetty module-init-tools mpc mpfr ncurses ncurses-base ncurses-libs netpbm net-snmp net-tools nspr nss nss-softokn nss-softokn-freebl nss-sysinit nss-util oniguruma openldap openssh openssl pam passwd patch pcre perl php pinentry pkgconfig popt postgresql ppl procps psmisc psutils pth python python3 readline recode redhat-rpm-config rpm rpm-build rpm-libs rsync ruby sed setup shadow shadow-utils shared-mime-info sharutils sqlite startup-notification strace stunnel sysfsutils system-config-keyboard systemd system-setup-keyboard systemtap sysvinit tar t-binutils tcl tcp_wrappers tcsh teckit telnet texinfo texlive t-gcc tix tk tmpwatch tzdata u-boot udev unzip usbutils usermode ustr util-linux util-linux-ng uuid vim wget which xinetd x-loader xz xz-libs yum yum-metadata-parser zip zlib
