Mass Branching - SOP
Description
Each new release we branch all active packages for the new release.
Action
Turn off email notification from the pkgdb. This can be done by turning email off in the pkgdb config file.
1. On puppet1, edit configs/web/applications/pkgdb-prod.cfg.erb 2. Make this change:
-mail.on = True +mail.on = False
3. Follow the Infrastructure Puppet SOP to push the changes to all the app servers (04-2008, app4 and app5.vpn)
The following work is performed on cvs-int
1. Disable logins to cvs 1. sudo crontab -u toshio -e 2. Comment out the gen-acl.sh cron job 3. CVSROOT=/cvs/pkgs cvs co CVSROOT 4. cp CVSROOT/admin/avail CVSROOT/admin/avail.bak 5. edit CVSROOT/admin/avail and comment out everything below the cvsadmins acl:
unavail | | rpms avail | @cvsadmin | rpms <!-- Fedora SPARC team --> <!-- Fedora IA64 team --> <!-- Fedora S390 team --> # # [...]
6. cvs commit your changes 2. Disable logging 1. CVSROOT=/cvs/pkgs cvs co CVSROOT 2. edit CVSROOT/loginfo and comment out all of the lines 3. cvs commit the change to the repository 3. Run pkgdb-client to add branches for packages. Notes: 1. you can specify multiple packages on a commandline. 2. You can pass a password in via stdin. 3. Possible script starter:
touch ~/pass chmod 0600 ~/pass cat ~/pass |./pkgdb-client -b F-9 --master devel PKGLIST1 cat ~/pass |./pkgdb-client -b F-9 --master devel PKGLIST2
4. Run pkgdb2branch.py to put the files and directories on the cvs server. 1. The --branch-for option should work for this::
./pkgdb2branch.py --branch-for=F-9
2. If for some reason that doesn't work, you can try this alternative:
cat pkglist.txt|./pkgdb2branch.py -c -
where pkglist.txt is a list of all the packages to branch.
Now turn everything back on.
As a final step, refresh the acls file with all the new F-9 branches:
/home/fedora/toshio/gen-acl.sh
Getting a List of Unbranched Packages
There are three things that list packages in releases. Each plays a part in this.
Koji keeps track of which packages should be branched and which are blocked. To get a list for say the F9 packages which are valid to branch:
koji list-pkgs --quiet --tag dist-f9 |awk '{print $1}' |sort > f9-branchlist
The pkgdb keeps a list of packages in branches. This can be found by running this sql query:
ssh db2 sudo -u postgres psql pkgdb select p.name from package as p where p.id not in (select p.id from package as p , packagelisting as pl, collection as c where c.id = pl.collectionid and p.id = pl.packageid and c.name = 'Fedora' and c.version = '9') order by p.name \g /var/tmp/not-in-f9
To get the final list of what needs branches, compare the koji output and if an entry from the koji output exists in the pkgdb output, it needs a branch.
Notes on Optimization
- If we can give a new flag to pkgdb's API to turn off mail on a per request basis we don't have to turn off pkgdb in order to branch.
- Try doing parallel pkgdb-client calls.
- If we could enhance the mail notification script for cvs to detect mass branching then we wouldn't need to close cvs during mass branching requests. (Currently, the rate for this is 3 pkgs/minute purely for the cvs portion (pkgdb information is cached at this point.))