Features/FixNetworkNameResolution

From FedoraProject

< Features(Difference between revisions)
Jump to: navigation, search
(Moved to FeatureReadyForFesco for approval (#986))
(Migrated to Changes/FixNetworkNameResolution)
(38 intermediate revisions by 4 users not shown)
Line 1: Line 1:
= Dualstack Networking =
+
= Fix Network Name Resolution =
  
 
== Summary ==
 
== Summary ==
  
Support proper dual stack networking. All major system services and
+
IPv4 and IPv6 host/service name resolution doesn't work very well in Fedora. The <code>getaddrinfo()</code> function in glibc returns wrong results in many cases. This feature aims to fix a bunch of name resolution bugs in glibc that prevent applications from fully using name resolution functions without doing a bunch of workarounds.
user applications should work with both IPv4 and IPv6. All of them
+
should also be able to cope with multiple addresses per hostname via
+
proper use of getaddrinfo().
+
  
 
== Owner ==
 
== Owner ==
Line 18: Line 15:
 
* Name [[User:tore|Tore Anderson]]
 
* Name [[User:tore|Tore Anderson]]
 
* Email: tore at fud.no
 
* Email: tore at fud.no
 +
 +
* Name [[User:Codonell|Carlos O'Donell]]
 +
* Email: carlos at redhat.com
  
 
== Current status ==
 
== Current status ==
  
* Targeted release: [[Releases/19|Fedora 19]]
+
* Targeted release: [[Releases/20|Fedora 20]]
* Last updated: 2013-01-02
+
* Last updated: 2013-03-14
* Percentage of completion: 0%
+
* Percentage of completion: 10%
  
 
== Detailed description ==
 
== Detailed description ==
  
Fedora supports dualstack global  networking. That means the computer with Fedora is connected to internet using both IPv4 and IPv6 protocols. But many important system services and applications either don't do IPv6, do it incorrectly, or don't cope with various network conditions.  
+
Currently the <code>getaddrinfo()</code> function doesn't work as it was designed. Many of its features are buggy and cannot be used without extensive workarounds. Many software packages are using <code>getaddrinfo()</code> with such workarounds. Many can trigger its failures. And many packages that don't use <code>getaddrinfo()</code> will be ported in the near future.
  
Unfortunately, while trying to improve IPv6 support, some IPv4 use cases became broken as well. That's why the goal of this feature is not only to support IPv4, but to support all possible real-world cases.
+
=== Rationale ===
  
Dualstack-ready software must cope with all possible scenarios including IPv4-only connectivity, IPv6-only connectivity and dual connectivity. The software must also cope with node-local (aka localhost) networking, which as been used by software for decades.
+
We are submitting this bug fixing effort as a Feature because:
  
Though it would be nice to have all applications in Fedora fixed to work in any of the scenarios, it is not feasible to test that. Therefore this feature is about major software used in servers, desktops and laptops. The list of such applications will be completed over the time.
+
* It is a high-impact change that will (positively) affect almost all networking software
 +
* Developers will be able to use <code>getaddrinfo()</code> without ugly workarounds for new code
 +
* We are going to publish guidelines for proper <code>getaddrinfo()</code> usage
 +
* Documentation for <code>getaddrinfo()</code> bugs will be available
 +
* Possible workarounds will be offered for backward compatibility
 +
* Comments and errata will be sent to standards organizations
 +
* We want to receive critical response during the whole process
 +
* It will be part of the next networking testweek
  
Bugs related to dualstack networking should be added to the following tracker bug:
+
=== Problem statement ===
  
http://bugzilla.redhat.com/show_bug.cgi?id=883152
+
The behavior of <code>getaddrinfo()</code> is often nonstandard, undocumented, surprising, or just plain wrong. We already identified a number of problems. The most prominent examples are here.
  
Also see: https://bugzilla.redhat.com/show_bug.cgi?id=ipv6blocker
+
* <code>getaddrinfo()</code> may return duplicate or even wrong addresses from /etc/hosts
 +
* <code>getaddrinfo()</code> with NULL servname may return duplicate addresses
 +
* <code>getaddrinfo()</code> with AI_PASSIVE may still return address list not suitable for <code>bind()</code>
 +
* <code>getaddrinfo()</code> with AI_ADDRCONFIG may fail to translate literal addresses
 +
* <code>getaddrinfo()</code> with AI_ADDRCONFIG may fail to resolve /etc/hosts addresses
 +
* <code>getaddrinfo()</code> with AI_ADDRCONFIG may send unwanted AAAA queries
 +
* <code>getaddrinfo()</code> has a bad choice of default flags
  
== Benefit to Fedora ==
+
Whether or not the problematic actually occurs depends on <code>/etc/hosts</code> configuration, <code>/etc/resolv.conf</code> configuration, <code>getaddrinfo()</code> input parameters, runtime kernel network interface configuration, and more. While testing the known bugs or reading the source code, more and more bugs are discovered.
  
One of Fedora's goals is to be a modern linux distribution that will work in modern environments. For this to be achieved, proper IPv4-only support and proper dualstack support is a MUST. IPv6-only use cases are rare but may become more common at any time. It's better to be prepared for those, too.
+
Bug reports related to <code>getaddrinfo()</code> can be found upstream:
  
== Scope ==
+
http://sourceware.org/bugzilla/buglist.cgi?quicksearch=getaddrinfo
  
Any software commonly used on day-to-day basis should be correctly handling IPv4-only, IPv6-only and dualstack use cases.
+
=== Affected software ===
  
Libraries:
+
The above problems affect software that wants to use <code>getaddrinfo()</code> to:
  
* glibc: [[Networking/NameResolution|name resolution]] must work properly
+
* Get parameters for <code>connect()</code> or <code>sendto()</code> to start communicating
* glib: provides networking API for applications
+
* Get parameters for <code>bind()</code> to listen on specific addresses
 +
* Build IP address based accesslists
 +
* Perform name resolution for other purposes
  
System services:
+
Although it would be nice to also test and fix all software in Fedora using <code>getaddrinfo()</code>, that is not feasible. Therefore we are going to concentrate on checking and fixing the GNU C library, checking and fixing the most important toolkits dealing with networking, and documenting a set of guidelines for daemons and application software.
  
* knfsd
+
Fedora bugs related to dualstack networking including name resolution problems should be added to the following tracker bug:
  
Applications:
+
http://bugzilla.redhat.com/show_bug.cgi?id=883152
  
* [http://bugzilla.redhat.com/show_bug.cgi?id=883155 Empathy refuses to connect to dualstack IRC servers temporarily available only through IPv4]
+
== Benefit to Fedora ==
  
Network testing tools:
+
Most Fedora installations are being used in networking environments. Many of them are laptop installations that connect to various networks or even  work offline. As name resolution is a critical part of networking experience, Fedora will benefit from being able offer reliability and avoid surprising and buggy behaviour.
  
* <code>ping</code> doesn't support IPv6 addresses nor hostnames
+
Bugs addressed by this feature page are especially tricky because an ordinary user is usually not able to properly report them, not to say analyze them. Especially bugs affecting plain old IPv4 services should be fixed and avoided in the future.
* <code>tcpdump</code> doesn't support IPv6 hostnames
+
  
Note: Network configuration tools are out of scope of this feature. See [[Networking#Fedora_feature_pages]] for other network-related features.
+
And there's a bonus. As all the documentation will be on Fedora Wiki, attracting attention of other members of the community. Together with other network-related efforts, Fedora could be seen as the leader in linux networking.
 +
 
 +
== Scope ==
 +
 
 +
Main focus:
 +
 
 +
* <code>getaddrinfo()</code>
 +
* Name service switch
 +
* DNS module
 +
* hosts module
 +
 
 +
Secondary libraries:
 +
 
 +
* GLib networking API
 +
* Qt networking API
 +
 
 +
Other output:
 +
 
 +
* Guidelines for daemons and applications
 +
* Backward compatibility guidelines
 +
* Documentation and hints for testing
 +
* Comments and/or errata for standards organizations
  
 
== How to test ==
 
== How to test ==
  
This is the most difficult part. We are going to combine the following approaches to testing:
+
* Reproducers will be always documented in the (usually upstream) bug reports
 +
* The best way to test <code>getaddrinfo()</code> is with Python's socket module
 +
* Qt and Glib can be also tested in Python
 +
* Some test programs in C could also be helpful
 +
* Lot of tests can be done with actual software using <code>getaddrinfo()</code> or one of the libraries
 +
* Everyday use can help, too
  
* Everyday use of Fedora in various network environments
+
Please note that we are going to test this feature as part of the next Fedora Network Test Week.
* Testing the tracked bug reports
+
* Specific tests, e.g. for <code>getaddrinfo()</code>
+
  
 
== User experience ==
 
== User experience ==
  
Users currently experience networking problems in various environments even during the most basic use of their system because of principal problems in the distribution. These problems would go away.
+
* Users will be less likely to see confusing and hard-to-reproduce errors
 +
* Some configurations will start working that didn't work before
 +
* Name resolution bugs in applications will be resolved more quickly
 +
* Developers will be able to concentrate on their real work
  
 
== Dependencies ==
 
== Dependencies ==
  
Not known.
+
Almost all development will be done via contributions upstream. Each library will have to be either updated or the fixes backported. Networking APIs are fairly independent from the rest of the libraries.
 +
 
 +
Libraries that ''may'' need updating or fixing:
 +
 
 +
* GNU C Library
 +
* GLib
 +
* Qt
  
 
== Contingency plan ==
 
== Contingency plan ==
  
This feature is about fixing all important parts of the distribution. We do not expect the required changes to negatively affect the rest of the packages. But if any problems occur, we are ready to help fixing them before the respective version is released. As a last resort, any problematic change can be reverted.
+
In case the fixes are not ready in time, all packages would continue to use
 +
workarounds that make them work with the buggy versions of glibc and/or other libraries.h
 +
 
 +
If, on the other hand, there are packages negatively affected by the changes, we will offer help with fixing the package and/or revert the changes for the next Fedora release. We do not expect any negative impact, though, and we are analysing any potential problems in advance.
 +
 
 +
We plan to publish a list of packages that might be directly affected by our modifications.
  
 
== Documentation ==
 
== Documentation ==
  
* [[Networking/Addressing]]
+
Currently, new documentation for name resolution is being drafted on the Fedora Wiki:
 +
 
 
* [[Networking/NameResolution]]
 
* [[Networking/NameResolution]]
 +
 +
Existing documentation:
 +
 +
* [http://pubs.opengroup.org/onlinepubs/9699919799/functions/getaddrinfo.html POSIX.1-2008]
 +
* RFC 3493 (informational)
 +
* RFC 2553 (obsolete informational)
 +
* [http://man7.org/linux/man-pages/man3/getaddrinfo.3.html Current getaddrinfo manpage]: mostly like RFC 3493
 +
* [http://www.akkadia.org/drepper/userapi-ipv6.html Ulrich Drepper: Userlevel IPv6 Programming Introduction]
  
 
== Release notes ==
 
== Release notes ==
  
Dualstack networking is fully supported and glibc and other packages will be tested for not including fundamental design flaws and important bugs regarding IPv4 and IPv6 support.
+
Name resolution for network host and service names now works as expected and in line with the standards and user needs. Daemons and applications will now resolve names correctly without
 +
extensive workarounds. Both IPv4 and IPv6 addresses are fully supported.
  
 
== Comments and discussions ==
 
== Comments and discussions ==
  
See [[Talk:Features/DualstackNetworking]]
+
See [[Talk:Features/FixNetworkNameResolution]]
  
[[Category:FeatureReadyForFesco]]
+
[[Category:FeaturePageIncomplete]]

Revision as of 12:53, 3 July 2013

Contents

Fix Network Name Resolution

Summary

IPv4 and IPv6 host/service name resolution doesn't work very well in Fedora. The getaddrinfo() function in glibc returns wrong results in many cases. This feature aims to fix a bunch of name resolution bugs in glibc that prevent applications from fully using name resolution functions without doing a bunch of workarounds.

Owner

Current status

  • Targeted release: Fedora 20
  • Last updated: 2013-03-14
  • Percentage of completion: 10%

Detailed description

Currently the getaddrinfo() function doesn't work as it was designed. Many of its features are buggy and cannot be used without extensive workarounds. Many software packages are using getaddrinfo() with such workarounds. Many can trigger its failures. And many packages that don't use getaddrinfo() will be ported in the near future.

Rationale

We are submitting this bug fixing effort as a Feature because:

  • It is a high-impact change that will (positively) affect almost all networking software
  • Developers will be able to use getaddrinfo() without ugly workarounds for new code
  • We are going to publish guidelines for proper getaddrinfo() usage
  • Documentation for getaddrinfo() bugs will be available
  • Possible workarounds will be offered for backward compatibility
  • Comments and errata will be sent to standards organizations
  • We want to receive critical response during the whole process
  • It will be part of the next networking testweek

Problem statement

The behavior of getaddrinfo() is often nonstandard, undocumented, surprising, or just plain wrong. We already identified a number of problems. The most prominent examples are here.

  • getaddrinfo() may return duplicate or even wrong addresses from /etc/hosts
  • getaddrinfo() with NULL servname may return duplicate addresses
  • getaddrinfo() with AI_PASSIVE may still return address list not suitable for bind()
  • getaddrinfo() with AI_ADDRCONFIG may fail to translate literal addresses
  • getaddrinfo() with AI_ADDRCONFIG may fail to resolve /etc/hosts addresses
  • getaddrinfo() with AI_ADDRCONFIG may send unwanted AAAA queries
  • getaddrinfo() has a bad choice of default flags

Whether or not the problematic actually occurs depends on /etc/hosts configuration, /etc/resolv.conf configuration, getaddrinfo() input parameters, runtime kernel network interface configuration, and more. While testing the known bugs or reading the source code, more and more bugs are discovered.

Bug reports related to getaddrinfo() can be found upstream:

http://sourceware.org/bugzilla/buglist.cgi?quicksearch=getaddrinfo

Affected software

The above problems affect software that wants to use getaddrinfo() to:

  • Get parameters for connect() or sendto() to start communicating
  • Get parameters for bind() to listen on specific addresses
  • Build IP address based accesslists
  • Perform name resolution for other purposes

Although it would be nice to also test and fix all software in Fedora using getaddrinfo(), that is not feasible. Therefore we are going to concentrate on checking and fixing the GNU C library, checking and fixing the most important toolkits dealing with networking, and documenting a set of guidelines for daemons and application software.

Fedora bugs related to dualstack networking including name resolution problems should be added to the following tracker bug:

http://bugzilla.redhat.com/show_bug.cgi?id=883152

Benefit to Fedora

Most Fedora installations are being used in networking environments. Many of them are laptop installations that connect to various networks or even work offline. As name resolution is a critical part of networking experience, Fedora will benefit from being able offer reliability and avoid surprising and buggy behaviour.

Bugs addressed by this feature page are especially tricky because an ordinary user is usually not able to properly report them, not to say analyze them. Especially bugs affecting plain old IPv4 services should be fixed and avoided in the future.

And there's a bonus. As all the documentation will be on Fedora Wiki, attracting attention of other members of the community. Together with other network-related efforts, Fedora could be seen as the leader in linux networking.

Scope

Main focus:

  • getaddrinfo()
  • Name service switch
  • DNS module
  • hosts module

Secondary libraries:

  • GLib networking API
  • Qt networking API

Other output:

  • Guidelines for daemons and applications
  • Backward compatibility guidelines
  • Documentation and hints for testing
  • Comments and/or errata for standards organizations

How to test

  • Reproducers will be always documented in the (usually upstream) bug reports
  • The best way to test getaddrinfo() is with Python's socket module
  • Qt and Glib can be also tested in Python
  • Some test programs in C could also be helpful
  • Lot of tests can be done with actual software using getaddrinfo() or one of the libraries
  • Everyday use can help, too

Please note that we are going to test this feature as part of the next Fedora Network Test Week.

User experience

  • Users will be less likely to see confusing and hard-to-reproduce errors
  • Some configurations will start working that didn't work before
  • Name resolution bugs in applications will be resolved more quickly
  • Developers will be able to concentrate on their real work

Dependencies

Almost all development will be done via contributions upstream. Each library will have to be either updated or the fixes backported. Networking APIs are fairly independent from the rest of the libraries.

Libraries that may need updating or fixing:

  • GNU C Library
  • GLib
  • Qt

Contingency plan

In case the fixes are not ready in time, all packages would continue to use workarounds that make them work with the buggy versions of glibc and/or other libraries.h

If, on the other hand, there are packages negatively affected by the changes, we will offer help with fixing the package and/or revert the changes for the next Fedora release. We do not expect any negative impact, though, and we are analysing any potential problems in advance.

We plan to publish a list of packages that might be directly affected by our modifications.

Documentation

Currently, new documentation for name resolution is being drafted on the Fedora Wiki:

Existing documentation:

Release notes

Name resolution for network host and service names now works as expected and in line with the standards and user needs. Daemons and applications will now resolve names correctly without extensive workarounds. Both IPv4 and IPv6 addresses are fully supported.

Comments and discussions

See Talk:Features/FixNetworkNameResolution