Docs/Drafts/MockSetupUsingLocalMirror

= Mock Setup Using Local Mirror =

Author: Parag Nemade

Revision: 0.4

Initial Draft: Friday Jun 30, 2006

Last Revised: Saturday Aug 21, 2006

Note: This page describes my personal mock setup using local mirror setup for FC6.

Setting Local Mirrors For Fedora Development version
Prior required steps are to have at least 10GB free space on your hard drive for setting Fedora development repository.

To setup local mirror, create following files

1)copy /etc/yum.repos.d/fedora-development.repo to /etc/yum.repos.d/mirror-fedora-development.repo

2)copy /etc/yum.repos.d/fedora-extras.repo to /etc/yum.repos.d/mirror-fedora-extras.repo

Say if you have /repo directory having minimum free space 10GB then download fedora-mirror.py from http://www.fedoraproject.org/wiki/Projects/fedora-mirror to /mirror_repo.

Change /srv/mirror to /mirror_repo in fedora-mirror.py file. For development version we need only development, extras and groups repository. We don't need any updates repository as its not exist.

Modified version of fedora-mirror.py
Final Modified version of fedora-mirror.py for development version is


 * 1) !/usr/bin/python


 * 1) License: GPL
 * 2) Author:  Rudolf Kastl
 * 3) default target: /mirror_repo
 * 4) how?: use config file and special mirror .repo files


 * 1) todo:


 * 1) code:
 * 2) - ConfigParser
 * 3) - logging
 * 4) - legacy and updates-testing awareness
 * 5) - fix hierachy into the fedora mirror dir scheme
 * 6) - use separate configs for each release
 * 7) - progressbars
 * 8) - eta (complete, per repo, per file(?!))
 * 9) support files/packaging/integration:
 * 10) - creation of specially crafted .repo files
 * 11) - cron integration
 * 12) - 0 conf apache serving as split package
 * 13) - mock config files as split package
 * 14) - yum localmirror config files as split package
 * 15) - discuss access rights on /mirror_repo


 * 1) special cases:
 * 2) - the fedora core repos differ in logic -> core repos have updates, updates-testing, legacy -> to be investigated
 * 3) - development is another special case not having updates updates-testing and legacy
 * 4) - other repos (3rd party) never have updates and updates-testing
 * 5) - SRPMS are another special case -> same for each arch of the same release
 * 1) - SRPMS are another special case -> same for each arch of the same release

import os

flavourlist=["core", "extras", "updates"] versionlist=["development"] archlist=["i386"] subflavourlist=["binary"]

from optparse import OptionParser
 * 1) block for parsing commandline switches

parser = OptionParser
 * 1) var can be accessed with options.filename

parser.add_option("-d", "--dest", dest="destination", help="Sync Destination", metavar="dir", default="/mirror_repo") parser.add_option("-n", "--newonly", action="store_true", dest="newonly", help="Sync Destination", default=False) parser.add_option("-t", "--testonly", action="store_true", dest="testonly", help="No Sync Transaction", default=False) parser.add_option("-q", "--quiet", action="store_false", dest="verbose", default=True, help="don't print status msgs to stdout")

(options, args) = parser.parse_args
 * 1) end bloc

targetlocation = options.destination

def syncMirror (version, flavour, arch, subflavour, targetlocation): print "syncing: Fedora " + version, flavour, arch, subflavour, "to", targetlocation


 * 1) handling the special cases

if subflavour == "binary": subflavourid = "" subflavourpath = "" elif subflavour == "source": subflavourid = "-"+subflavour subflavourpath = "/SRPMS" else: subflavourid = "-"+subflavour subflavourpath = "-"+subflavour


 * 1) devel is special

if version == "development": repoid="mirror-"+flavour+"-"+version+subflavourid else: repoid="mirror-"+flavour+subflavourid

if subflavour == "source": targetdir=targetlocation+"/fedora/"+version+"/"+flavour+subflavourpath else: targetdir=targetlocation+"/fedora/"+version+"/"+flavour+"/"+arch+subflavourpath# handling the newonly option of reposync

if options.newonly == True: newonly = " -n" else: newonly = ""

syncdistro="mirrorreleasever="+version+" mirrorbasearch="+arch+" /usr/bin/reposync"+newonly+" -r "+repoid+" -p "+targetdir
 * 1) building the command for syncing a single repo


 * 1) for testing we just print instead of sync

if options.testonly == True: print syncdistro else:
 * 1) create potentially missing paths

try: if os.path.isdir(targetdir) == False: os.makedirs(targetdir) except: print "Error: Cannot create targetdirectory:", targetdir

os.system(syncdistro)


 * 1) main loop

for version in versionlist: for flavour in flavourlist: for arch in archlist: for subflavour in subflavourlist:
 * 1) stuff that just doesent happen

if version == "development" and flavour == "updates": continue if version == "development" and flavour == "legacy": continue if version == "development" and flavour == "updates-testing": continue else: syncMirror(version, flavour, arch, subflavour, targetlocation)

Creating Directories for Local Repository Path
Now start downloading all development repository to /mirror_repo by giving

python fedora-mirror.py

Once all core and extras development repository downloaded, Now move core and extras repository to parent directory /mirror_repo.

fedora-mirror.py will download core repository at

/mirror_repo/fedora/development/core/i386/mirror-core-development

fedora-mirror.py will download extras repository at

/mirror_repo/fedora/development/extras/i386/mirror-extras-development

Create following directories

/mirror_repo/fedora/development

/mirror_repo/fedora/development/i386/

/mirror_repo/fedora/development/i386/core

/mirror_repo/fedora/development/i386/extras

/mirror_repo/fedora/development/i386/groups

Setting up core repository directory
Then create clean i386 repository by doing following

mv /mirror_repo/fedora/development/core/i386/mirror-core-development/* /mirror_repo/fedora/development/i386/core

cd /mirror_repo/fedora/development/i386/core

createrepo.

Setting up extras repository directory
mv /mirror_repo/fedora/development/extras/i386/mirror-extras-development /mirror_repo/fedora/development/i386/extras

cd /mirror_repo/fedora/development/i386/extras

createrepo.

Setting up groups repository directory
cd /mirror_repo/fedora/development/i386/groups

create minimal.xml and add its contents as

 <!DOCTYPE comps PUBLIC "-//Red Hat, Inc.//DTD Comps info//EN" "comps.dtd">

build true Minimal Install bash bzip2 coreutils cpio diffutils fedora-release gcc gcc-c++ gzip make patch perl rpm-build redhat-rpm-config sed tar unzip buildsys-macros

download buildsys-build and buildsys-macors rpms under /mirror_repo/fedora/development/i386/groups

wget http://buildsys.fedoraproject.org/buildgroups/development/i386/buildsys-build-0.5-1.noarch.rpm

wget http://buildsys.fedoraproject.org/buildgroups/development/i386/buildsys-build-0.5-3.fc6.noarch.rpm

wget http://buildsys.fedoraproject.org/buildgroups/development/i386/buildsys-macros-6-2.fc6.noarch.rpm

wget http://buildsys.fedoraproject.org/buildgroups/development/i386/buildsys-macros-6-2.fc6.src.rpm

Final contents of directory groups will look as

ls /mirror_repo/fedora/development/i386/groups

minimal.xml buildsys-macros-6-2.fc6.noarch.rpm buildsys-macros-6-2.fc6.src.rpm buildsys-build-0.5-1.noarch.rpm buildsys-build-0.5-3.fc6.noarch.rpm

now do createrepo -g minimal.xml. under /mirror_repo/fedora/development/i386/groups

Thus you will find all repository set under /mirror_repo/fedora/development/i386.

Mock Setup for Local Mirror
I have setup mock environment for development(FC6) version. My mock configuration for i386 is /etc/mock/fedora-devel-i386-core.cfg

import os config_opts['root'] = 'fedora-development-i386-core' config_opts['target_arch'] = 'i386'

config_opts['yum.conf'] = """ [main] cachedir=/var/cache/yum debuglevel=1 reposdir=/dev/null logfile=/var/log/yum.log retries=20 obsoletes=1 gpgcheck=0 assumeyes=1

[core] name=core baseurl=file:///mirror_repo/fedora/development/i386/core

[updates-released] name=updates baseurl=file:///mirror_repo/fedora/development/i386/core

[groups] name=groups baseurl=file:///mirror_repo/fedora/development/i386/groups

[extras] name=extras baseurl=file:///mirror_repo/fedora/development/i386/extras

[local] name=local baseurl=http://extras64.linux.duke.edu/plague-results/fedora-5-extras enabled=0

"""

Building packages in Mock Environment
To build packages in mock you need to go through http://fedoraproject.org/wiki/PackageMaintainers/MockTricks

Troubleshooting for fedora-mirror.py
If you face any problems using fedora-mirror.py script then you can use /usr/bin/reposync command to download binary RPMS for extras repository.

Core repository can be setup by copying FC6 installation DVD/CDROMs to /mirror_repo/fedora/development/i386/core. Detailed Procedure to do this can be found at http://www.city-fan.org/tips/YumRepoFromImages

reposync command without any options will download extras repository. Just move all those RPMS to /mirror_repo/fedora/development/i386/extras and use  command

Thus still you can download and create local repository without fedora-mirror.py script.