From Fedora Project Wiki

< User:Till

Revision as of 17:33, 5 March 2010 by Till (talk | contribs) (→‎Workflow: add package set calculation)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Idea for a workflow to speedup the availability of updates:


Workflow

  • Use one directory for all RPMS that are used in updates/updates-testing called updates-packages/
  • As soon as a new update is created, the affected RPMS are added to updates-packages/
  • once a day the repository metadata for all packages in updates and updates-testing is created and it only references the respective packages, the package set is determined by using the same algorithms that are currently used (e.g. latest package from koji for the matching tag and some multilib magic)
  • Once this metadata is synced to the mirrors, all rpms that are not included in the previous metadata are removed from updates-packages/
  • every N hours, repository metadata for all packages added since to the respective repo prior to the previous creation of the full metadata is created and provides as a new repo $repo-recent, which only contains additional metadata, but not a new copy of rpms

reasoning

  • This ensures that rpms can be hardlinked and do not need to be synced again when they move from updates-testing to updates
  • The create of the $repo-recent metadata will happen much faster, because only a small amount of packages need to be read, e.g. at maximum the rpms added within the last 48 hours if the full metadata is created every day
  • Since the rpms are pushed to the mirrors before the metadata is ready, the new metadata can be used without problems immediately
  • To allow to only mirror e.g. updates but not updates testing, for both repos a directory that hardlinks the respective rpms can be used. The hardlinks need to be updated after every update state change, e.g. a request to push a package to stable needs to add a new hardlinkg to the updates repository