Infrastructure/Mirroring/zh-cn

Fedora 项目在全球拥有超过 200 个志愿镜像站点 点击查看 ，这有助于分发 Fedora. 我们十分感谢他们！

联系我们

 * 邮件列表: mirror-list (仅发布镜像变更声明) 和 mirror-list-d (讨论专用)
 * IRC 频道:
 * 管理员:

镜像大约有多大？
请仔细阅读.
 * 详见：http://dl.fedoraproject.org/pub/DIRECTORY_SIZES.txt

Export Compliance
By downloading Fedora software, you acknowledge that you understand all of the following: Fedora software and technical information may be subject to the U.S. Export Administration Regulations (the “EAR”) and other U.S. and foreign laws and may not be exported, re-exported or transferred (a) to any country listed in Country Group E:1 in Supplement No. 1 to part 740 of the EAR (currently, Cuba, Iran, North Korea, Sudan & Syria); (b) to any prohibited destination or to any end user who has been prohibited from participating in U.S. export transactions by any federal agency of the U.S. government; or (c) for use in connection with the design, development or production of nuclear, chemical or biological weapons, or rocket systems, space launch vehicles, or sounding rockets, or unmanned air vehicle systems. You may not download Fedora software or technical information if you are located in one of these countries or otherwise subject to these restrictions. You may not provide Fedora software or technical information to individuals or entities located in one of these countries or otherwise subject to these restrictions. You are also responsible for compliance with foreign law requirements applicable to the import, export and use of Fedora software and technical information.

如何搭建一个公共镜像？
搭建公共镜像非常简单，而且将越来越容易. 您的网站拥有足够的带宽和磁盘空间来承担负载，这就是我们的全部要求. 每个Fedora发行版都会消耗200GB以上的磁盘空间，并且下载者可能会耗尽您的全部带宽. 镜像站点有至少100Mbps的带宽连接到互联网，现有许多站点是基于千兆或更高的带宽工作的. 以 Fedora 8 发行版为例，其在主服务器上消耗的总空间（也是镜像站点要消耗的空间）达到了1.1TB并且仍在增长. 1-2TB的空间适合于建立长期镜像. 这是基于硬链接的空间概念，如果您不能使用硬链接（例如，使用AFS），您将需要更多的磁盘空间. 实际磁盘空间消耗在此处给出

http://dl.fedoraproject.org/pub/DIRECTORY_SIZES.txt

如何搭建一个私人镜像？
私人镜像是那些只能被内部人员访问的某些组织或者机构专有的镜像，例如公司,学校等等. 这些镜像是为了在机构内部加快 Fedora的分发而被开设，当然，这些镜像所处的网络对当地的流量消耗远少于连接因特网的.

私人镜像 behave similarly to public mirrors, with a few exceptions: You may also find it more beneficial to run an IntelligentMirror instead of a full rsync mirror. In this way, only the updates your local users actually need will be cached on your local mirror, saving you the bandwidth from downloading updates you don't actually need.
 * Private mirrors are never listed in the MirrorManager publiclist pages.
 * Private mirrors cannot pull from the master Fedora download servers. They must pull from another listed public mirror.
 * Private mirrors should include IP netblocks in their MirrorManager configuration. This allows your network-local users to be automatically redirected to your mirror.  You may list IP netblocks (e.g. 18.0.0.0/8), or if your network is NAT'd, the hostname of your NAT gateway.
 * Private mirrors are not crawled by the MirrorManager web crawler. As a corollary:
 * Private mirrors must run report_mirror to inform the MirrorManager database of their content. If you don't run report_mirror, your clients will not be automatically redirected.

镜像管理器：Fedora 项目的镜像管理系统
镜像管理器跟踪所有的镜像而不需要很多的人工的文件编辑. 镜像站点的管理员一点要确保  脚本和   在每次  之后被运行以更新镜像管理数据库.

Fedora 账户系统

 * You must have an account in the Fedora Account System . (More info also at  .) You are not required to sign the Contributors License Agreement to merely mirror Fedora content, but you must do so if you wish to contribute to other aspects of Fedora.
 * If a public mirror, you must send an email to  introducing yourself, stating you would like to become a mirror, your IP address, your location (country), and your outbound bandwidth available for the mirror.  Private mirrors are encouraged to send a similar note.
 * You must subscribe to mirror-list and (optionally) mirror-list-d  (discussion) to be notified of new releases.

在 MirrorManager 中注册
$ dig txt 1.1.166.143.asn.routeviews.org @archive.routeviews.org ;; ANSWER SECTION: 1.1.166.143.asn.routeviews.org. 86400 IN TXT	"3614" "143.166.0.0" "16" Here, the answer is in the TXT record, the first value, 3614.
 * 使用您的 FAS 账户登录 mirrormanager ；
 * 创建一个新站点；
 * create a new Host, and sign up that host for the Categories of content you'll carry, any other site administrators you want, your site's IP addresses used for our Access Control List, and the other details listed there if applicable to you
 * Please run  after each rsync run.
 * You may list your site's IP address ranges (Netblocks). Clients coming from an IP address within your netblock will be automatically redirected to your mirror for any content you carry.
 * You may list your site's BGP Autonomous System Number (ASN). Clients on your ASN will be automatically redirected to your mirror for any content you carry.  One way to lookup up your ASN is to query it from the routeviews.org DNS servers.  It is like a PTR record lookup, but at a specific server.  For example, to look up 143.166.1.1, type:

搭建
The only sane way to do mirroring is to use. Note the options  (hardlinks), ,   and   are required to ensure your mirror content stays valid even during a new rsync run, until all the new data is available.

rsync -vaH --exclude-from=${EXCLUDES} --numeric-ids --delete --delete-after --delay-updates \ rsync://dl.fedoraproject.org/fedora-enchilada ${LOCAL_DIR}


 * You may exclude any content you desire, such as architectures, using an EXCLUDES file.


 * Please pull from one of the Tier 1 mirrors. See Infrastructure/Mirroring/Tiering .  Instead of using one of the Tier 1 servers, you may wish to pull from another fast mirror that's closer to you.  Contact the respective mirror admins to be added to their ACL.


 * You should sync shortly after 0800 UTC (when rawhide is pushed), 1400 UTC (when bitflips occur), and another 3-5 times per day (updates are manually released).

45 */6 * * * perl -le 'sleep rand 1800' && bash -l ~/mirror-fedora > /dev/null
 * You should add some random value to the start time of your rsync jobs, to even out load on the upstream mirrors. A cron line might look like:


 * If you are using rsync 3.0 or higher, you can use the  option instead of , which is reported to provide faster performance.

Running report_mirror
MirrorManager includes a tool,  which can upload to the mirror database that you completed a run and what content you've got. This makes generating the yum mirrorlists and all other pages much much simpler. Please run  after every rsync job completes.

yum install mirrormanager-client

or get the files directly from the git tree report_mirror files. Or it can be obtained using git:

git clone git://git.fedorahosted.org/git/mirrormanager or git clone http://git.fedorahosted.org/git/mirrormanager/

You need both report_mirror and report_mirror.conf, and must edit report_mirror.conf to include the content you're carrying and the path to that content on your disk.

可用内容
The available content modules by rsync, and their point in the directory tree are:

Suggested rsync modules
These other modules exist for legacy purposes, and should be avoided by current mirrors.

Red Hat Enterprise Linux Beta
RHEL beta releases are also managed via MirrorManager. Fedora's Tier 1 mirrors may sync these modules.

DVDs, CDs, and the exploded trees
When a new release is available, it can be bandwidth-efficient to download only the ISOs first (say, the DVD ISOs), then explode those into the directory structure, then run a full normal rsync run. This lets you avoid downloading the same RPMs twice (both on ISOs and as plain RPMs). There's a tool somewhere to help do this.

Regular hardlink runs
While the Fedora release maintainers try to keep as little redundant packaging around as possible, there are some duplicate packages in the tree. For example, when a Fedora Test release comes out, the package set included there looks remarkably like that of the development tree from a few days before. By copying the development tree over into the new Test directory before starting your rsync run, and using, you can avoid downloading all that content a second time.

In addition, it's good practice to run a tool like  on your tree occasionally (say, weekly), to ensure as much of your tree as possible is hardlinked.

Pre-Release: Copying Development tree to new release directory
In the days leading up to a release, either test or final, the development tree will stop taking new packages, and will closely resemble what winds up in the new release. As a mirror, you can avoid downloading content that already is in your copy of the development tree that matches what's in the release tree by copying those packages using hardlinks, such as: cp -lr fedora/linux/development/13/i386/os fedora/linux/releases/13/Fedora/i386/ cp -lr fedora/linux/development/13/x86_64/os fedora/linux/releases/13/Fedora/x86_64/ cp -lr fedora/linux/development/13/source fedora/linux/releases/13/Fedora/

and then start the rsync process, which will clean up any changes and fix up the timestamps.

Rsync Configuration (sample)
Larger mirrors, like kernel.org, have slightly custom front-ends to rsync (mainly so that they can have a single rsync instance and have multiple ip based vhost configuration files) That said what follows is a sample rsync configuration file for public syncing (this is not intended for private pre-bitflip mirroring)

[fedora] comment        = Fedora - RedHat community project path           =  exclude        = lost+found/ read only      = true max connections = 100 lock file      = /var/run/rsyncd-mirrors.lock uid            =  gid            =  transfer logging = yes timeout        = 900 ignore nonreadable = yes dont compress  = *.gz *.tgz *.zip *.z *.Z *.rpm *.deb *.bz2 refuse options = checksum

Things to explicitly note:
 * The path above should be a full path to your fedora directory
 * You should *really* want to leave this read-only
 * Make sure your uid/gid are set to public users, not to the user that you run as your sync agent. If you set this to the user who does your syncs you will be inadvertently giving the public full pre-bitflip access.
 * Make sure you have the 'refuse options' set to checksum, your server will be *MUCH* happier with this set, as it will prevent public users from performing a checksum run against you. This can be incredibly I/O abusive, so should not be available to the general public.

Keepalives
HTTP Keepalives should be enabled on your mirror server to speed up client downloads. By default, Fedora's Apache httpd package has keepalives disabled. They should be enabled, with a timeout of at least 2 seconds (the default of 15 seconds might be too high for a heavily loaded mirror server, but 2 seconds is sufficient and appropriate for yum).

KeepAlive On KeepAliveTimeout 2 MaxKeepAliveRequests 100 Other http servers such as lighttpd have keepalives enabled by default.

Caching of metadata
We don't want caching proxy servers between our mirrors and our end user systems to cache our yum repository metadata. So, add explicit metadata handling. (Suggested by the OpenSUSE download redirector.)

 Header set Cache-Control "must-revalidate" ExpiresActive On      ExpiresDefault "now" 

Redirecting ISO downloads to FTP
While no longer a recommended practice, the following mod_rewrite rules will force all *.iso files to be downloaded via FTP. In this example HEAD requests are not redirected, so the MirrorManager crawler is not disrupted.

RewriteCond    %{REQUEST_METHOD} GET RewriteRule    ^(.*\.iso)$ ftp://myserver/$1  [L,R=301]

Content Types
ISO and RPM files should be served using MIME Content-Type: application/octet-stream. In Apache, this can be done inside a VirtualHost or similar section:

 AddType application/octet-stream .iso AddType application/octet-stream .rpm 

Limiting Download Accelerators
Download accelerators will try to open the same file many times, and request chunks, hoping to download them in parallel. This can overload heavily loaded mirror servers, especially on release day. Here are some tricks to thwart such activities.

To limit connections to ISO dirs by some amount per IP:

 MaxConnPerIP 6 

To block ranged requests as this is what download accelerators do indeed:

RewriteEngine on RewriteCond %{HTTP:Range} [0-9] $ RewriteRule \.iso$ / [F,L]

Similar things can be done with iptables and the recent module, which might give you a little more ability to control what is being done, either by limiting new connections or by dropping 50% of a users packets.

Logging Partial Content Downloads
Partial content can be logged correctly using apache:


 * 1) this includes actual counts of actual bytes received (%I) and
 * 2) sent (%O); this requires the mod_logio module to be loaded.

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" %I %O \"%{User-Agent}i\"" combined

Pre-bitflip mirroring
Several days before each public release, the content will be staged to the master mirror servers, but with restricted permissions on the directories (generally mode 0750), specifically, not world readable.

Mirror servers should have several different user/group accounts on their server, for running the different public services. Typically you find:
 * HTTP server runs as user apache, group apache
 * FTP server runs as user ftp, group ftp
 * RSYNC server runs as user rsync, group rsync
 * a user account for downloading content from the masters (e.g. user mirror, group mirror).

The user account used to download content from the masters must be not be the same as the HTTP, FTP, or RSYNC server accounts. This guarantees that content downloaded with permissions 0750 will not be made available via your public servers yet.

On the morning of the public release, the permissions on the directories on the master servers will change to 0755 - world readable. This is called the bitflip.

Mirrors may either rsync one more time to pick up these new permissions (but won't have to download all the data again), or preferably, can schedule a batch job to bitflip:

$ echo "chmod a+rx /pub/fedora/linux/releases/9" | at '14:45 UTC May 13 2008'

Serving content to other mirrors
Tier 1 mirrors will necessarily need to share content to Tier 2 mirrors before the bitflip. This is done by running another instance of the rsync daemon, on a different port (e.g. 874), with an Access Control List to prevent public downloads, running as a user in the same group as downloaded the content (e.g. group mirror). This could be user mirror, group mirror, who has group read/execute permissions on the still-private content.

Tier 1 mirrors have a tendency to use different authentication methods for granting access to these non-public downloads, they vary from maintaining IP based ACL's to assigning username/password combinations to mirrors wishing to sync from them. Each method has advantages / disadvantages, the IP list is 'simpler' from a mirrormanager perspective as mirrormanager can give you the list of IP's but from an automation standpoint can be more difficult (as rsync's configuration file does not allow that ACL list to be stored in a separate file). Username / passwords can be more versatile as sites mirroring can change IPs without notifying you, but it's easier for those credentials to leak out and get miss-used.

全球镜像分布地图
http://fedoraproject.org/maps/mirrors.png

每日更新.

感谢
本页包含由 MaxMind 旗下产品 GeoLite 的数据. 详情请前往：http://www.maxmind.com/ 查看.