From Fedora Project Wiki


(link for [8a])
(clear away non-used stuff as we did not issue an "emergency" #140)
Line 1: Line 1:
= Breaking News on the Infrastructure Outage =
This special issue of FWN covers the infrastructure problems as reported in FWN#139 "General Outage of Fedora Infrastructure" all that was known[1] was that the problems became obvious to a wide audience on August 13th.
An update[2] was posted by [[PaulFrields|Paul Frields]] on 18 August which listed the services which had returned to normal and were expected to return to normal soon. Public speculation latched on[3][4] to the fact that the SSH keys of "fedorahosted" had changed. Most guesses used this as evidence that something similar to the recent 2008 Debian OpenSSL vulnerabilities (not be confused with the 2003 Debian Project compromise[5] which was due to a 0-day kernel exploit or the 2006 compromise[6]) had occurred.  FAS holders received an email asking them to reset their passwords as a precautionary measure which further heightened suspicions that something similar to the SSL problem had occurred. Some confusion prevailed[8] on @fedora-devel as to whether it was possible to trust the new key fingerprint on the website. JimMeyering added[8a] a useful post which explained how to change from using a DSA ssh key to an RSA ssh key. Overall there was a surprisingly low level of public discussion of the problem and it was not until 18 August that some complaints about the lack of information were expressed[9] on @fedora-list.
[5] A key developer's machine was compromised due to a kernel exploit[5a] and then password sniffers were installed which provided the attacker(s) with root access to at least one key Debian server ( This was used as a staging post to install another sniffer and a chain of two more servers were compromised. As as result of modifications made to the one of the kernels it started OOPsing and investigations of this revealed the problem.
[7] Metasploit has an excellent writeup on the topic here:
On 22 August [[PaulFrields|Paul Frields]] posted[10] an announcement that stated that an intrusion had been detected on Fedora Project machines including the package signing machine. An extensive audit suggests that the passphrase to the key was not sniffed yet it was decided to treat the intrusion as having potentially compromised the key. The result of this has been that all the keys are being reissued.
Date  Fri, 22 Aug 2008 8:00 AM
From  "Paul W. Frields" <>
To  "fedora-announce-list" <>
Subject  Infrastructure report, 2008-08-22 UTC 1200
Last week we discovered that some Fedora servers were illegally
accessed. The intrusion into the servers was quickly discovered, and the
servers were taken offline.
Security specialists and administrators have been working since then to
analyze the intrusion and the extent of the compromise as well as
reinstall Fedora systems. We are using the requisite outages as an
opportunity to do other upgrades for the sake of functionality as well
as security. Work is ongoing, so please be patient. Anyone with
pertinent information relating to this event is asked to contact
One of the compromised Fedora servers was a system used for signing
Fedora packages. However, based on our efforts, we have high confidence
that the intruder was not able to capture the passphrase used to secure
the Fedora package signing key. Based on our review to date, the
passphrase was not used during the time of the intrusion on the system
and the passphrase is not stored on any of the Fedora servers.
While there is no definitive evidence that the Fedora key has been
compromised, because Fedora packages are distributed via multiple
third-party mirrors and repositories, we have decided to convert to new
Fedora signing keys. This may require affirmative steps from every
Fedora system owner or administrator. We will widely and clearly
communicate any such steps to help users when available.
Among our other analyses, we have also done numerous checks of the
Fedora package collection, and a significant amount of source
verification as well, and have found no discrepancies that would
indicate any loss of package integrity. These efforts have also not
resulted in the discovery of additional security vulnerabilities in
packages provided by Fedora.
Our previous warnings against further package updates were based on an
abundance of caution, out of respect for our users. This is also why we
are proceeding with plans to change the Fedora package signing key. We
have already started planning and implementing other additional
safeguards for the future. At this time we are confident there is little
risk to Fedora users who wish to install or upgrade signed Fedora
In connection with these events, Red Hat, Inc. detected an intrusion of
certain of its computer systems and has issued a communication to Red
Hat Enterprise Linux users which can be found at This communication
states in part, "Last week Red Hat detected an intrusion on certain of
its computer systems and took immediate action. While the investigation
into the intrusion is on-going, our initial focus was to review and test
the distribution channel we use with our customers, Red Hat Network
(RHN) and its associated security measures. Based on these efforts, we
remain highly confident that our systems and processes prevented the
intrusion from compromising RHN or the content distributed via RHN and
accordingly believe that customers who keep their systems updated using
Red Hat Network are not at risk. We are issuing this alert primarily for
those who may obtain Red Hat binary packages via channels other than
those of official Red Hat subscribers."
It is important to note that the effects of the intrusion on Fedora and
Red Hat are *not* the same. Accordingly, the Fedora package signing key
is not connected to, and is different from, the one used to sign Red Hat
Enterprise Linux packages. Furthermore, the Fedora package signing key
is also not connected to, and is different from, the one used to sign
community Extra Packages for Enterprise Linux (EPEL) packages.
We will continue to keep the Fedora community notified of any updates.
Thank you again for your patience.

Revision as of 14:20, 23 August 2008