Infrastructure/Mirroring/pt-br

In other languages: English (en) 中文（中国大陆）‎ (zh-cn) português (pt) português do Brasil (pt-br) русский (ru) [edit]

O Fedora tem a sorte de ter "mais de 200 voluntários [1] globalmente", o que ajuda a distribuir o software Fedora para usuários finais. Agradecemos muito os nossos mirrors e os administradores do sistema.

Fedora Export Compliance/Customs Information

By downloading Fedora software, you acknowledge that you understand all of the following: Fedora software and technical information may be subject to the U.S. Export Administration Regulations (the “EAR”) and other U.S. and foreign laws and may not be exported, re-exported or transferred (a) to a prohibited destination country under the EAR or U.S. sanctions regulations (currently Cuba, Iran, North Korea, Sudan, Syria, and the Crimea Region of Ukraine, subject to change as posted by the United States government); (b) to any prohibited destination or to any end user who has been prohibited from participating in U.S. export transactions by any federal agency of the U.S. government; or (c) for use in connection with the design, development or production of nuclear, chemical or biological weapons, or rocket systems, space launch vehicles, or sounding rockets, or unmanned air vehicle systems. You may not download Fedora software or technical information if you are located in one of these countries or otherwise subject to these restrictions. You may not provide Fedora software or technical information to individuals or entities located in one of these countries or otherwise subject to these restrictions. You are also responsible for compliance with foreign law requirements applicable to the import, export and use of Fedora software and technical information.

Fedora software in source code and binary code form are publicly available and are not subject to the EAR in accordance with §742.15(b).

---

Comunicação

• Listas de Discussão: mirror-admins ** NOTA 2017-03-21 Nova localização da lista **
• IRC: #fedora-admin[?] no Freenode
• Mudanças administrativas: envie e-mail para mirror-admin@fedoraproject.org

Antes de decidir se tornar um mirror

Quais são as estimativas de tamanho?

Leia o arquivo de texto DIRECTORY_SIZES.txt com cuidado. Se você pretende alocar a quantidade de espaço necessário para o mirror do Fedora, continue lendo.

Como alguém pode se tornar um mirror publico?

Tornar-se um mirror público oficial é fácil. Tudo o que solicitamos é que seu site tenha largura de banda e espaço em disco suficientes para lidar com a carga.

Espaço em disco

Each Fedora release can consume upwards of 250GB of disk space. As of the Fedora 26 release, the total space consumed on the master server, thus what a mirror could consume, is 1.5TB and growing. Older releases are periodically archived, so a 3-4TB volume would be most appropriate for a long-term mirror. This content is hardlinked; if you can't hardlink (e.g. you're on AFS), you'll need much more disk space. Actual space consumed is noted in DIRECTORY_SIZES.txt.

Bandwidth

Downloaders can consume as much bandwidth as you've got. Mirror sites have at least a 100Mbit/sec connection to the Internet, many have Gigabit/sec, 10 Gigabit/sec, or larger pipes. 100Mbit/sec is the minimum required for new mirrors in countries with adequate mirror coverage already. We can make exceptions for new mirrors in countries that have few mirrors. Connections to Internet2, National Lambda Rail, GEANET2, RedIRIS, or other such high speed research and educational networks are always appreciated.

Como alguém pode fazer um mirror privado?

Os mirrors privados são mirrors que residem inteiramente dentro de uma organização (empresa, escola, etc.) e só podem ser acessados ​​por membros dessa organização. Eles existem para acelerar a distribuição do Fedora dentro de uma organização.

Os mirrors privados se comportam de forma semelhante aos mirrors públicos, com poucas exceções:

• Os mirrors privados nunca estão listados nas páginas de listagem pública do MirrorManager.
• Os mirrrors privados não podem extrair dos servidores de download principais do Fedora. Eles devem puxar de outro mirror público listado.
• Os mirrrors privados devem incluir netblocks de IP na configuração MirrorManager. Isso permite que seus usuários locais da rede sejam redirecionados automaticamente para o seu mirror. Você pode listar IP netblocks (por exemplo, 18.0.0.0/8), ou se sua rede é NAT'd, o nome do host do seu gateway NAT.
• Os mirrors privados não são rastreados pelo rastreador da Web MirrorManager. Assim sendo:
• Mirrors privados devem ser executados report_mirror para informar o banco de dados MirrorManager do seu conteúdo. Se você não executar o report_mirror, seus clientes não serão redirecionados automaticamente.

MirrorManager: o sistema de gerenciamento de mirror Fedora

O software [2] mantém o controle de todos os mirrors sem exigir muita edição manual de arquivos de texto. A instância do Fedora deste aplicativo está em https://admin.fedoraproject.org/mirrormanager

Mirror Status

O MirrorManager acompanha todos os mirrors com varreduras regulares de conteúdo de todos os mirrors. O conteúdo dos mirrors é verificado pelo rastreador que faz a varredura/rastreio de mirrors usando HTTP ou RSYNC. Recomendamos encarecidamente fornecer um URL RSYNC (pelo menos para o nosso rastreador), pois o rastreamento via RSYNC é muito mais rápido, pois requer apenas uma conexão de rede única para cada categoria espelhada. Se apenas o acesso HTTP para o rastrear o mirror do Fedora for possível, é importante habilitar HTTP Keepalives para reduzir o número de conexões de rede necessárias.

Além disso, os administradores dos espelhos devem garantir que o script report_mirror disponível com o mirrormanager-client seja executado após cada rsync executar para atualizar o Mirrormanager base de dados.

Inscrição

Fedora Account System

• Você deve ter uma conta no Fedora Account System (mais informações também em AccountSystem). Você não precisa assinar o Contrato de Licença de Contribuintes para limitar o conteúdo do Fedora, mas você deve fazê-lo se desejar contribuir com outros aspectos do Fedora.
• Se for um espelho público, você deve enviar um e-mail para mirror-admin@fedoraproject.org apresentando-se, afirmando que gostaria de se tornar um espelho, seu endereço IP, sua localização (país) e sua largura de banda de saída disponível para o espelho. Os espelhos privados são encorajados a enviar uma nota semelhante.
• Você deve se inscrever em mirror-adminpara ser notificado sobre novos lançamentos.

Registrar-se no MirrorManager

• Logue no MirrorManager usando os dados da sua conta FAS.
• Crie um novo site (um site é um grupo de Hosts pertencentes à mesma organização).
• Crie um novo Host e inscreva-se nesse host para as Categorias de conteúdo que você carregará (na seção opcionalmente Categorias' Carrried), qualquer outro administrador do site que você deseja, os endereços IP do seu site usado para nossa lista de controle de acesso, lista de países que estão autorizados a acessar seu mirror, e os outros detalhes listados lá, se aplicável.
  • Você pode listar os intervalos de endereço IP do seu site (Netblocks). Os clientes provenientes de um endereço IP dentro do seu netblock serão redirecionados automaticamente para o seu mirror para qualquer conteúdo que você carregue.
  • Você pode listar o Número de Sistema Autônomo BGP do seu site (ASN). Os clientes no seu ASN serão redirecionados automaticamente para o seu espelho para qualquer conteúdo que você carregue. Uma maneira de pesquisar o seu ASN é consultá-lo dos servidores de DNS routeviews.org. É como uma pesquisa de registro PTR, mas em um servidor específico. Por exemplo, para procurar 143.166.1.1, digite:
    

    $ dig txt 1.1.166.143.asn.routeviews.org @archive.routeviews.org
     ;; SEÇÃO DE RESPOSTAS:
    1.1.166.143.asn.routeviews.org.	86400 IN TXT	"3614" "143.166.0.0" "16"

    Aqui, a resposta está no registro TXT, o primeiro valor, 3614.

  • Para que um mirror seja listado no MirrorManager para a categoria de conteúdo que você selecionou para transportar, você deve "adicionar" pelo menos um URL através do qual a categoria de conteúdo em seu mirror pode ser acessada (em 'Categorias Carregadas '). Os URLs que você adiciona podem ser do tipo HTTP, HTTPS e rsync. A instância do MirrorManager do Fedora não suporta URLs de FTP e, portanto, não é possível adicionar mais um URL de FTP.
• Por favor, rode report_mirror depois de cada rsync executar. (Nota: isto não é necessário se você usa o Quick-fedora-mirror, veja abaixo para mais detalhes.)

Mirroring

• Please use the quick-fedora-mirror zsh script if at all possible. This script uses rsync and some informational files on the mirrors to allow you to only sync those files you need and saves lots of time and file seeking. Note: make sure that the mirror you intend to use provides the fedora-buffet rsync module, as otherwise the script will not work.

• Please pull from one of the Tier 1 mirrors. Instead of using one of the Tier 1 servers, you may wish to pull from another fast mirror that's closer to you. Contact the respective mirror admins to be added to their ACL.

• You may exclude any content you desire, such as architectures or releases, using an EXCLUDES file or --exclude parameter.

Tools to avoid

Please don't use tools like 'lftp mirror' to mirror content from the master mirrors. It's places a heavy load on our master mirrors and is slow for both you and us. Please use quick-fedora-mirror instead.

Mirror Frequency

• If you are using quick-fedora-mirror you can sync as often as every 10minutes in a loop. If there is no new content the script should return almost immediately. Make sure to use locking so you are only running one instance at a time.

• If you are not using quick-fedora-mirror you should sync a few times a day and consider switching.

Running report_mirror

Note: quick-fedora-mirror can also check in for you and has no need of the report_mirror package/conf. To enable it, set the correct values for CHECKIN_SITE and CHECKIN_PASSWORD variables. In most cases, setting CHECKIN_HOST is not necessary. The following applies only if you are not using quick-fedora-mirror, and, as said above, you should consider switching.

MirrorManager includes a tool, report_mirror which can upload to the mirror database that you completed a run and what content you've got. This makes generating the yum mirrorlists and all other pages much much simpler. Please run report_mirror after every rsync job completes. You can get this tool via dnf (in Fedora 22 or later):

dnf install mirrormanager2-client

or yum (in Fedora 21 or earlier, RHEL and CentOS):

yum install mirrormanager2-client

or get the files directly from the official Git repository:

git clone https://github.com/fedora-infra/mirrormanager2.git

You need both report_mirror and report_mirror.conf, and must edit report_mirror.conf to include the content you're carrying and the path to that content on your disk.

Available content

The available content modules by rsync, and their point in the directory tree are:

Suggested rsync modules

rsync module	Description	path on master server	Comments
fedora-buffet	Fedora -- the whole buffet (all you can eat)	/pub	Please use this if you can, it provides the all current Fedora content, including pre-bitflip content. This is open to specific mirrors by request. Mirrors participating in our tiering should use this. Mirrors syncing both fedora-enchilada and fedora-archive should use this, as we can now hardlink across both of those trees under fedora-buffet0.
fedora-enchilada	Fedora -- the whole enchilada	/pub/fedora	Please use this if you can, it provides the all current Fedora content, including pre-bitflip content. This is open to specific mirrors by request. Mirrors participating in our tiering should use this.
fedora-epel	Extra Packages for Enterprise Linux	/pub/epel	Please use this to mirror EPEL
fedora-archive	Historical Fedora releases	/pub/archive	Fedora Core 1-6 and Extras 3-6, and obsolete releases 7 and higher
fedora-secondary	Fedora Secondary Arches	/pub/fedora-secondary
fedora-alt	Fedora Other	/pub/alt

These other modules exist for legacy purposes, and should be avoided by current mirrors.

rsync module	Description	path on master server	Comments
fedora-linux-releases	Fedora Linux Releases	/pub/fedora/linux/releases
fedora-linux-development	Fedora Linux Development (Rawhide)	/pub/fedora/linux/development
fedora-linux-updates	Fedora Linux Updates	/pub/fedora/linux/updates

Rsync Configuration (sample)

Larger mirrors, like kernel.org, have slightly custom front-ends to rsync (mainly so that they can have a single rsync instance and have multiple ip based vhost configuration files) That said what follows is a sample rsync configuration file for public syncing (this is not intended for private pre-bitflip mirroring)

[fedora]
        comment         = Fedora - RedHat community project
        path            = <path to your fedora directory>
        exclude         = lost+found/
        read only       = true
        max connections = 100
        lock file       = /var/run/rsyncd-mirrors.lock
        uid             = <user id (numeric, or textual) of an anonymous style user who should have read access>
        gid             = <group id (numeric, or textual) of an anonymous style user who should have read access>
        transfer logging = yes
        timeout         = 900
        ignore nonreadable = yes
        dont compress   = *.gz *.tgz *.zip *.z *.Z *.rpm *.deb *.bz2
        refuse options = checksum

Things to explicitly note:

• The path above should be a full path to your fedora directory
• You should *really* want to leave this read-only
• Make sure your uid/gid are set to public users, not to the user that you run as your sync agent. If you set this to the user who does your syncs you will be inadvertently giving the public full pre-bitflip access.
• Make sure you have the 'refuse options' set to checksum, your server will be *MUCH* happier with this set, as it will prevent public users from performing a checksum run against you. This can be incredibly I/O abusive, so should not be available to the general public.

HTTPd Configuration

Keepalives

HTTP Keepalives should be enabled on your mirror server to speed up client downloads. By default, Fedora's Apache httpd package has keepalives disabled. They should be enabled, with a timeout of at least 2 seconds (the default of 15 seconds might be too high for a heavily loaded mirror server, but 2 seconds is sufficient and appropriate for yum).

KeepAlive On
KeepAliveTimeout 2
MaxKeepAliveRequests 100

Other http servers such as lighttpd have keepalives enabled by default.

Caching of metadata

We don't want caching proxy servers between our mirrors and our end user systems to cache our yum repository metadata. So, add explicit metadata handling. (Suggested by the OpenSUSE download redirector.)

   <LocationMatch "\.(xml|xml\.gz|xml\.asc|sqlite)$">
       Header set Cache-Control "must-revalidate"
       ExpiresActive On
       ExpiresDefault "now"
   </LocationMatch>

Content Types

ISO and RPM files should be served using MIME Content-Type: application/octet-stream. In Apache, this can be done inside a VirtualHost or similar section:

<VirtualHost *:80>
AddType application/octet-stream .iso
AddType application/octet-stream .rpm
</VirtualHost>

Limiting Download Accelerators

Download accelerators will try to open the same file many times, and request chunks, hoping to download them in parallel. This can overload heavily loaded mirror servers, especially on release day. Here are some tricks to thwart such activities.

To limit connections to ISO dirs by some amount per IP:

<IfModule mod_limitipconn.c>
MaxConnPerIP 6
</IfModule>

To block ranged requests as this is what download accelerators do indeed:

RewriteEngine on
RewriteCond %{HTTP:Range} [0-9] $
RewriteRule \.iso$ / [F,L] 

Similar things can be done with iptables and the recent module, which might give you a little more ability to control what is being done, either by limiting new connections or by dropping 50% of a users packets.

Logging Partial Content Downloads

Partial content can be logged correctly using apache:

# this includes actual counts of actual bytes received (%I) and
# sent (%O); this requires the mod_logio module to be loaded.

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" %I %O \"%{User-Agent}i\"" combined

Pre-bitflip mirroring

Several days before each public release, the content will be staged to the master mirror servers, but with restricted permissions on the directories (generally mode 0750), specifically, not world readable.

Mirror servers should have several different user/group accounts on their server, for running the different public services. Typically you find:

• HTTP server runs as user apache, group apache
• FTP server runs as user ftp, group ftp
• RSYNC server runs as user rsync, group rsync
• a user account for downloading content from the masters (e.g. user mirror, group mirror).

The user account used to download content from the masters must be not be the same as the HTTP, FTP, or RSYNC server accounts. This guarantees that content downloaded with permissions 0750 will not be made available via your public servers yet.

On the morning of the public release, the permissions on the directories on the master servers will change to 0755 - world readable. This is called the bitflip.

Mirrors may either rsync one more time to pick up these new permissions (but won't have to download all the data again), or preferably, can schedule a batch job to bitflip:

$ echo "chmod a+rx /pub/fedora/linux/releases/9" | at '14:45 UTC May 13 2008'

Serving content to other mirrors

Tier 1 mirrors will necessarily need to share content to Tier 2 mirrors before the bitflip. This is done by running another instance of the rsync daemon, on a different port (e.g. 874), with an Access Control List to prevent public downloads, running as a user in the same group as downloaded the content (e.g. group mirror). This could be user mirror, group mirror, who has group read/execute permissions on the still-private content.

Tier 1 mirrors have a tendency to use different authentication methods for granting access to these non-public downloads, they vary from maintaining IP based ACL's to assigning username/password combinations to mirrors wishing to sync from them. Each method has advantages / disadvantages, the IP list is 'simpler' from a mirrormanager perspective as mirrormanager can give you the list of IP's but from an automation standpoint can be more difficult (as rsync's configuration file does not allow that ACL list to be stored in a separate file). Username / passwords can be more versatile as sites mirroring can change IPs without notifying you, but it's easier for those credentials to leak out and get miss-used.

Outras características do MirrorManager

Estatísticas

Trilhas do MirrorManager the number of accesses to the mirror list e quebra por país, arquitetura e repositório.

Mirror Maps

The map of all public mirrors é atualizado diariamente; se não exibir os dados corretos, tente novamente mais tarde.

Reconhecimento

Este produto inclui GeoLite data criado por MaxMind.