Mass Branching SOP

From FedoraProject

(Difference between revisions)
Jump to: navigation, search
(Add the new pkgdb commands)
(cvs logging and cvs branching procedure)
Line 25: Line 25:
 
=== CVS ===
 
=== CVS ===
  
The following work is performed on cvs-int
+
The following work is performed on cvs1
  
1. Disable logins to cvs
+
==== Disable logins to cvs ====
1. sudo crontab -u toshio -e
+
<ol>
2. Comment out the gen-acl.sh cron job
+
  <li><code>sudo crontab -u toshio -e</code></li>
3. CVSROOT=/cvs/pkgs cvs co CVSROOT
+
  <li>Comment out the gen-acl.sh cron job</li>
4. cp CVSROOT/admin/avail CVSROOT/admin/avail.bak
+
  <li><code>CVSROOT=/cvs/pkgs cvs co CVSROOT</code></li>
5. edit CVSROOT/admin/avail and comment out everything below the cvsadmins acl:
+
  <li><code>cp CVSROOT/admin/avail CVSROOT/admin/avail.bak</code></li>
 +
  <li>edit CVSROOT/admin/avail and comment out everything below the cvsadmins acl:
 
<pre>
 
<pre>
 
unavail | | rpms
 
unavail | | rpms
 
avail | @cvsadmin | rpms
 
avail | @cvsadmin | rpms
<!-- Fedora SPARC team
+
## Fedora SPARC team
-->
+
#avail | @fedora-sparc | rpms
<!-- Fedora IA64 team
+
## Fedora IA64 team
-->
+
#avail | @fedora-ia64 | rpms
<!-- Fedora S390 team
+
## Fedora S390 team
-->
+
#avail | @fedora-s390 | rpms
 
#
 
#
 
#
 
#
 +
#avail | dwmw2,@provenpackager | rpms/0xFFFF
 
[...]  
 
[...]  
 
</pre>
 
</pre>
6. cvs commit your changes
+
  </li>
2. Disable logging
+
  <li>cvs commit your changes</li>
1. CVSROOT=/cvs/pkgs cvs co CVSROOT
+
</ol>
2. edit CVSROOT/loginfo and comment out all of the lines
+
 
3. cvs commit the change to the repository
+
==== Disable logging ====
3. Run pkgdb-client to add branches for packages. Notes:
+
<ol>
1. you can specify multiple packages on a commandline.
+
  <li><code>CVSROOT=/cvs/pkgs cvs co CVSROOT</code></li>
2. You can pass a password in via stdin.
+
  <li>edit CVSROOT/loginfo and comment out all of the lines</li>
3. Possible script starter:
+
  <li><cvs commit the change to the repository</li>
 +
 
 +
==== Make cvs branches ====
 +
Run pkgdb2branch.py to put the files and directories on the cvs server. The --branch-for option was designed with this use case in mind::
 
<pre>
 
<pre>
touch ~/pass
+
./pkgdb2branch.py --branch-for=F-10
chmod 0600 ~/pass
+
cat ~/pass |./pkgdb-client -b F-9 --master devel PKGLIST1
+
cat ~/pass |./pkgdb-client -b F-9 --master devel PKGLIST2
+
 
</pre>
 
</pre>
4. Run pkgdb2branch.py to put the files and directories on the cvs server.
+
 
1. The --branch-for option should work for this::
+
If for some reason that doesn't work, you can try this alternative:
<pre>
+
./pkgdb2branch.py --branch-for=F-9
+
</pre>
+
2. If for some reason that doesn't work, you can try this alternative:
+
 
<pre>
 
<pre>
 
cat pkglist.txt|./pkgdb2branch.py -c -
 
cat pkglist.txt|./pkgdb2branch.py -c -
Line 74: Line 72:
 
Now turn everything back on.
 
Now turn everything back on.
  
As a final step, refresh the acls file with all the new F-9 branches:
+
As a final step, refresh the acls file with all the new F-10 branches:
 
<pre>
 
<pre>
 
/home/fedora/toshio/gen-acl.sh
 
/home/fedora/toshio/gen-acl.sh

Revision as of 00:49, 7 November 2008

Contents

Mass Branching - SOP

Description

Each new release we branch all active packages for the new release.

Action

PackageDB

Mass branching in the pkgdb is the first step. It should be done near the time that the cvs branches are created so as not to confuse packagers. However, it does not cause an outage so it should be done ahead of time.

Mass branching is performed by a cvsadmin either from the command line or by pushing a button on a web page. The web page method is to browse to https://admin.fedoraproject.org/pkgdb/collections/ login, and then click the "Branch All" button.

The command line method is to run pkgdb-client --mass-branch -b F-10

Whichever method you choose, the command will start the mass branch process on the server and then return with a message that the branching is in progress.

When the branching is finished, you will receive an email that tells you of any unbranched packages as well as all the packages that were branched.

If something fails spectacularly, it is safe to try mass branching again at a later time. If only a few cleanups a re needed it might be better to do that with the regular branch commands.

CVS

The following work is performed on cvs1

Disable logins to cvs

  1. sudo crontab -u toshio -e
  2. Comment out the gen-acl.sh cron job
  3. CVSROOT=/cvs/pkgs cvs co CVSROOT
  4. cp CVSROOT/admin/avail CVSROOT/admin/avail.bak
  5. edit CVSROOT/admin/avail and comment out everything below the cvsadmins acl:
    unavail | | rpms
    avail | @cvsadmin | rpms
    ## Fedora SPARC team
    #avail | @fedora-sparc | rpms
    ## Fedora IA64 team
    #avail | @fedora-ia64 | rpms
    ## Fedora S390 team
    #avail | @fedora-s390 | rpms
    #
    #
    #avail | dwmw2,@provenpackager | rpms/0xFFFF
    [...] 
    
  6. cvs commit your changes

Disable logging

  1. CVSROOT=/cvs/pkgs cvs co CVSROOT
  2. edit CVSROOT/loginfo and comment out all of the lines
  3. <cvs commit the change to the repository
  4. Make cvs branches

    Run pkgdb2branch.py to put the files and directories on the cvs server. The --branch-for option was designed with this use case in mind::

    ./pkgdb2branch.py --branch-for=F-10
    

    If for some reason that doesn't work, you can try this alternative:

    cat pkglist.txt|./pkgdb2branch.py -c -
    

    where pkglist.txt is a list of all the packages to branch.

    Now turn everything back on.

    As a final step, refresh the acls file with all the new F-10 branches:

    /home/fedora/toshio/gen-acl.sh
    

    Getting a List of Unbranched Packages

    There are three things that list packages in releases. Each plays a part in this.

    Koji keeps track of which packages should be branched and which are blocked. To get a list for say the F9 packages which are valid to branch:

    koji list-pkgs --quiet --tag dist-f9 |awk '{print $1}' |sort > f9-branchlist
    

    The pkgdb keeps a list of packages in branches. This can be found by running this sql query:

    ssh db2
    sudo -u postgres psql pkgdb
    select p.name from package as p where p.id not in (select p.id from
    package as p , packagelisting as pl, collection as c
    where c.id = pl.collectionid and p.id = pl.packageid and
    c.name = 'Fedora' and c.version = '9')
    order by p.name \g /var/tmp/not-in-f9
    

    To get the final list of what needs branches, compare the koji output and if an entry from the koji output exists in the pkgdb output, it needs a branch.

    Notes on Optimization

    • If we can give a new flag to pkgdb's API to turn off mail on a per request basis we don't have to turn off pkgdb in order to branch.
    • Try doing parallel pkgdb-client calls.
    • If we could enhance the mail notification script for cvs to detect mass branching then we wouldn't need to close cvs during mass branching requests. (Currently, the rate for this is 3 pkgs/minute purely for the cvs portion (pkgdb information is cached at this point.))