From Fedora Project Wiki

< QA‎ | Networking

 
(33 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== IPv4 connectivity ==
== Connectivity ==
 
The following table lists basic IPv4 connectivity configurations. There are numerous less common configuration including a combination of public and private addresses and multiple public addresses.


{|
{|
!rowspan=2| Connectivity
!rowspan=2| IPv4 connectivity
!colspan=4| Address
!colspan=4| Address
!rowspan=2| Details
!rowspan=2| Details
|-
|-
! Public
! <abbr title="Public">Pub</abbr>
! Private
! <abbr title="Private">Priv</abbr>
! Link-local
! <abbr title="Link-local">LL</abbr>
! Loopback
! <abbr title="Loopback">Lo</abbr>
|-
|-
| Global
| Global
Line 44: Line 46:
| no
| no
| yes
| yes
| Host cannot use services in the
| Host has only a loopback address.
|}
|}


Note: It looks like link-local IPv4 addresses never get automatically configured in default Fedora 23 Workstation with NetworkManager. This is not necessarily a problem as it is much better to rely on ubiquitous IPv6 link-local addresses nowadays.
The following table lists basic IPv6 connectivity configurations. There are numerous less common configuration including a combination of public and ULA addresses or public or ULA address without a link-local one.
 
== IPv6 connectivity ==


{|
{|
!rowspan=2| Connectivity
!rowspan=2| IPv6 connectivity
!colspan=4| Address
!colspan=4| Address
!rowspan=2| Details
!rowspan=2| Details
Line 80: Line 80:
| yes
| yes
| yes
| yes
| Client only has a link-local address (IPv6LL).
| Host has only link-local (IPv6LL) and loopback addresses.
|-
|-
| None
| None
Line 87: Line 87:
| no
| no
| yes
| yes
| Client doesn't have any addresses except loopback.
| Host has only a loopback address.
|-
|-
| Disabled
| Disabled
Line 96: Line 96:
| IPv6 has been disabled in the kernel.
| IPv6 has been disabled in the kernel.
|}
|}
In most cases you will need only global or link-local connectivity. The latter is typically configured automatically when you connect via IPv6, the former needs to be present in the network. You can overcome this limitation by using an IPv6 tunnel like the one provided by http://www.tunnelbroker.net/. Any of the setups above can be mimicked on an isolated testing network by configuring a router advertisement daemon, an optional DHCPv6 server and a DNS server. You can use dnsmasq to provide all those services at once.
There are specific situations that aren't covered in this list like global addresses without global connectivity but those need to be addressed carefully in test cases.
=== How to get IPv6 connectivity ===
Only some networks offer IPv6 addresses and only some of them offer actual ''global'' connectivity using those addresses. If you are not that lucky to have such a network at hand, you can use one of the many services that offer IPv6 subnets via IPv6 tunnels. For virtually all practical purposes tunnelled connectivity is equivalent to native one. Getting ''local'' connectivity is not different from IPv4. On the other hand, you get ''link-local'' IPv6 connectivity automatically when connected to a physical ethernet or wireless ethernet network and it is usually kept through the physical connection lifetime unlike IPv4 link-local addresses. It is normally enough to connect two client machines using a cable to get it.
TODO As far as I know NetworkManager also supports disconnected IPv6 configuration where you don't even have link-local addresses but that needs to be tested first.
There are specific situations that aren't covered in this list like global addresses without global connectivity but those need to be addressed carefully in test cases.
== IPv4 and IPv6 connectivity combinations ==


The following table lists common or somehow interesting combinations of IPv4 and IPv6 connectivity.
The following table lists common or somehow interesting combinations of IPv4 and IPv6 connectivity.
Line 116: Line 102:
! IPv4 \ IPv6
! IPv4 \ IPv6
! Global
! Global
! LL Only
! Local
! Link-local
! None
! None
! Disabled
|-
|-
! Global
! Global
| Dual-stack server
| Dual-stack server
| IPv4 only server
|
| Disabled IPv6
| IPv4 server
|
|
|-
|-
! Masqueraded
! Masqueraded
| [[../Client_software#Dual-stack_to_dual-stack|Dual-stack workstation]]
| [[../Client_software#Dual-stack_to_dual-stack|Dual-stack workstation]]
| [[../Client_software#IPv4_to_dual-stack_with_lost_AAAA_answer|IPv4 only workstation]]
|
| IPv4 only workstation on PPP
| [[../Client_software#IPv4_to_dual-stack_with_lost_AAAA_answer|IPv4 workstation]]
| IPv4/PPP workstation
| Disabled IPv6 workstation
|-
! Local
|
| Private network
|
|
|
|-
! Link-local
|
|
| Dual-stack zeroconf
| IPv4 zeroconf
|
|-
|-
! None
! None
| IPv6 only
| IPv6 node
| IPv6LL only
|
| No connectivity
| IPv6 zeroconf
| Isolated node
|
|}
 
The following table provides details for selected combinations from the previous table.
 
{|
! Scenario
! Details
|-
| Dual-stack server
| Server available on public IPv4 and IPv6 addresses.
|-
| Dual-stack workstation
| Workstation connecting via both IPv4 and IPv6 Internet services.
|-
| IPv4 server
| Server available on IPv4 with IPv6 limited to link-local and loopback communication.
|-
| IPv4 workstation
| Workstation connecting via IPv4 with IPv6 limited to link-local and loopback communication.
|-
| IPv4/PPP workstation
| Mobile workstation connecting via IPv4 with IPv6 limited to loopback communication as IPv4-only PPP won't assign <abbr title="IPv6 link-local">IPv6LL</abbr>.
|-
| Disabled IPv6
| Workstation connecting via IPv4 with IPv6 disabled in the kernel.
|-
| IPv6 node
| Server available on public IPv6 address with IPv4 limited to loopback communication.
|-
| Private network
| Workstation or server on an isolated private network using IPv4 private addresses and IPv6 <abbr title="Unique Local Addresses">ULA</abbr> for local communication.
|-
| Dual-stack zeroconf
| Workstation or appliance in a zero configuration network after a failed DHCPv4 attempt.
|-
| IPv6 zeroconf
| Workstation or appliance in a zero configuration network on a system with <abbr title="IPv4 link-local">IPv4LL</abbr> disabled.
|-
| IPv4 zeroconf
| Old-school node in a zero configuration network.
|-
| Isolated node
| Disconnected node limited to loopback IPv4 and IPv6 communication.
|}
|}


The above table only contains selected combinations that are of some greater interest. For example all combinations with private addresses (ULA in case of IPv6) are left out.
=== How to get connectivity for everyday use ===


=== Details for selected connectivity combinations ===
Most home and office setups offer masqueraded IPv4 connectivity. In addition they often offer global IPv6 connectivity, often firewalled. Some corporate setups, on the other hand, don't offer connectivity to remote hosts except via a proxy server. Global and unmasqueraded IPv4 and IPv6 connectivity for servers is typically available in datacenters.


TODO
IPv4 and IPv6 loopback addresses are assigned automatically at boot time. IPv6 link-local addresses are specific in that they are automatically assigned by the kernel when an interface gets connected to a network on the link layer. On the other hand, IPv4 link-local addresses only serve as a fallback for unsuccessful DHCPv4 attempt.


=== How to get connectivity for everyday use ===
Note: It looks like link-local IPv4 addresses never get automatically configured in default Fedora 23 Workstation with NetworkManager. This is not necessarily a problem as it is much better to rely on ubiquitous IPv6 link-local addresses nowadays.


Most home and office setups offer masqueraded IPv4 connectivity. In addition they often offer global IPv6 connectivity, often firewalled. Some corporate setups, on the other hand, don't offer connectivity to remote hosts except via a proxy server.
==== IPv6 tunnels ====


Global and unmasqueraded IPv4 and IPv6 connectivity for servers is typically available in datacenters.
Many networks still only have IPv4 connectivity. You can overcome this limitation by configuring an IPv6 tunnel to a network with IPv6 connectivity. You can use various tunnel technologies to achieve that. There are also services like http://www.tunnelbroker.net/ that offer free tunnelled connectivity with IPv6 subnets.


=== How to emulate connectivity in a virtual test environment ===
=== How to emulate connectivity in a virtual test environment ===
Line 151: Line 202:
Using two virtual machines, one can emulate any type of IPv4 and IPv6 connectivity by picking up an address block intended for documentation and testing (<code> 192.0.2.0/24</code>, <code>198.51.100.0/24</code>, <code>203.0.113.0/24</code> and <code>2001:DB8::/32</code>) and setting up DHCPv4, RA, DHCPv6 and DNS services on one of the virtual machines, while using the other as a client node.
Using two virtual machines, one can emulate any type of IPv4 and IPv6 connectivity by picking up an address block intended for documentation and testing (<code> 192.0.2.0/24</code>, <code>198.51.100.0/24</code>, <code>203.0.113.0/24</code> and <code>2001:DB8::/32</code>) and setting up DHCPv4, RA, DHCPv6 and DNS services on one of the virtual machines, while using the other as a client node.


==== Router ====
==== Router with automatic configuration ====


Server always needs to have a valid static configuration. For simple testing, you can configure the server temporarily using iproute package.
Server always needs to have a valid static configuration. For simple testing, you can configure the server temporarily using iproute package.


<pre>
<pre>
# ip address add 192.0.2.1/24 dev eth1
interface=eth1
# ip address add 2001:db8::1/64 dev eth1
 
ip address add 192.0.2.1/24 dev $interface
ip address add 2001:db8::1/64 dev $interface
ip link set $interface up
</pre>
</pre>


Line 163: Line 217:


<pre>
<pre>
# dnsmasq -d \
dnsmasq -d \
     --dhcp-range=lan,192.0.2.100,192.0.2.200,10m \
     --dhcp-range=lan,192.0.2.100,192.0.2.200,10m \
     --enable-ra --dhcp-range=lan,2001:db8::1:0,2001:db8::2:0,10m
     --enable-ra --dhcp-range=lan,2001:db8::1:0,2001:db8::2:0,10m
</pre>
</pre>


==== Workstation ====
==== Automatically configured node ====


A proper workstation installation will get configured automatically.  
A proper workstation installation will get configured automatically using NetworkManager. A server can do exactly the same especially if the router is configured with static address assignments via DHCP. Or the server can be configured manually.


On the other hand you can test the server configuration using dhcpcd command from the dhcpcd package. Use the following command. If you leave out  
On the other hand you can test the server configuration using dhcpcd command from the dhcpcd package. Use the following command. If you leave out  
Line 176: Line 230:


<pre>
<pre>
# dhcpcd -d -T eth0
dhcpcd -d -T eth0
</pre>
 
Note: There are other non-default configuration tools including <code>systemd-networkd</code> and <code>connman</code> but those are out of scope of this document for now.
 
=== How to test network scenarios using Linux network namespaces ===
 
==== A pair of veth devices connecting two namespaces ====
 
<pre>
clientns="test-client"
clientif=$clientns
serverns="test-server"
serverif="$serverns
 
 
# setup the network namespaces
ip netns add $serverns
ip netns add $clientns
 
# add veth device
ip link add dev $serverif type veth peer name $clientif
 
# add the veth device endpoints to the appropriate network namespace
ip link set $clientif netns $clientns
ip link set $serverif netns $serverns
 
# enable the loopback interfaces
ip netns exec $clientns ip link set lo up
ip netns exec $serverns ip link set lo up
</pre>
 
==== Router with automatic configuration ====
 
<pre>
# configure the static ip addresses on the interface where dnsmasq will be run
ip netns exec $serverns ip address add 192.0.2.1/24 dev $serverif
ip netns exec $serverns ip address add 2001:db8::1/64 dev $serverif
 
# activate the interface
ip netns exec $serverns ip link set $serverif up
 
# start dnsmasq as the DHCP server and DNS resolver
ip netns exec $serverns \
    dnsmasq -d \
    --dhcp-range=lan,192.0.2.100,192.0.2.200,10m \
    --enable-ra --dhcp-range=lan,2001:db8::1:0,2001:db8::2:0,10m
</pre>
 
==== Automatically configured node ====
 
<pre>
ip netns exec test-node \
    dhcpcd -d -T eth0
</pre>
 
<pre>
ip netns exec test-node \
    ping router.example.net
</pre>
</pre>


Line 185: Line 297:
== DNS configuration ==
== DNS configuration ==


A dual-stack server is announced in DNS by at least one A record and at least one AAAA record. IPv4 only and IPv6 only servers have only A record or AAAA record respectively. Configurations with multiple A and AAAA records are interesting as well and so are configurations with SRV and other intermediate records.
A client connects to a service using IPv4 or IPv6 addresses. In many cases the client retrieves a domain name from user configuration or user input and uses that domain name to query DNS for the IP addresses. The most universal translation of a service domain to an IP address is through a combination of SRV and A/AAAA records. First the service is translated to a set of host names using the SRV record, then each host name is translated to a set of IPv4/IPv6 addresses using the A/AAAA records. In many cases indirection using SRV records is not supported and the host name must also be used as the service names. In some cases a different record (MX, NS, ...) is used for the indirection.
 
The intermediate records (SRV, MX, NS, ...) are protocol neutral and vary by service. At least one A and at least one AAAA service is needed for proper operation of a dual-stack service. IPv4 only and IPv6 only services only have one or more A or AAAA record respectively. Configuration
with multiple A and AAAA records (especially in combination with multiple SRV or other intermediate records) are useful for DNS based failover.


Key requirements:
Key requirements:


* The server must have a known host name
* Domain name of the service is known (from user configuration or input)
* Authoritative DNS servers must provide the right A/AAAA records
* Optional: Authoritative DNS zone provides a list of host names for the service domain name
* They should be accessible over both IPv4 and IPv6
* Authoritative DNS zone provides correct A/AAAA records for the host domain name
* Recursive DNS server must be able to get the records
* Service is available on addresses in A/AAAA records
* Local host must be able to query it and get all the records
* Recursive DNS server must be able to retrieve he above records
* Local host must be able tu query the DNS server and connect to at least some of the addresses
 
Possible reactions to DNS requests:
 
{|
! Reply type
! DNS rcode
! Meaning
! Details
|-
| DNS
| NOERROR
| Data
| Reply with one or more records of the requested type.
|-
| DNS
| NXDOMAIN
| No such domain
| Reply saying that there are no records at all for the domain name.
|-
| DNS
| NODATA
| No data
| Reply saying that there are records for the domain name but not of the requested type.
|-
| ICMP
| –
| Destination unreachable
| The server isn't available for communication.
|-
| –
| –
| No reply at all
| The server failed to reply or the reply was dropped on the firewall.
|}


The test environment often requires a local recursive and authoritative DNS server. In most cases a dnsmasq instance or any other forwarding nameserver with some authoritative capabilities will do the job. For testing, domains can be chosen under <code>example.net</code> and <code>example.com</code> subtrees.
The test environment often requires a local recursive and authoritative DNS server. In most cases a dnsmasq instance or any other forwarding nameserver with some authoritative capabilities will do the job. For testing, domains can be chosen under <code>example.net</code> and <code>example.com</code> subtrees.
=== DNS server accessibility ===
Domain name servers serve different roles. Authoritative servers provide zone data configured by service administrators. Recursive servers retrieve zone data from authoritative servers on behalf of the clients.
Recursive servers are part of the client infrastructure. In a dual-stack network they need to be be available on both IPv4 and IPv6 to also serve single-stack clients using any of the protocols and addresses for both protocols should be announced using automatic configuration protocols. In setups with additinal forwarding DNS servers that do not perform full recursion this applies to those DNS servers that actually serve clients.
The following table contains common combinations of dual-stack, IPv4 only and IPv6 only network network clients and the respective requirements on recursive DNS service availability.
{|
!colspan=3| Client IPv4/IPv6 support
!colspan=2| RDNSS requirements
|-
! Dual-stack
! IPv4 only
! IPv6 only
! IPv4
! IPv6
|-
| yes/no
| yes
| yes
| yes
| yes
|-
| yes/no
| yes
| no
| yes
| optional
|-
| yes/no
| no
| yes
| optional
| yes
|-
| yes
| no
| no
|colspan=2| at least one of them
|}
Authoritative servers are part of the service infrastructure. For dual-stack services they need to be available on both protocols in order to serve single-stack recursive DNS servers. For single-stack services they need to be available on the respective protocol.
{|
!rowspan=2| Service connectivity
!colspan=2| DNS records
!colspan=2| Authoritative DNS
|-
! A
! AAAA
! IPv4
! IPv6
|-
| Dual-stack
| yes
| yes
| yes
| yes
|-
| IPv4 only
| yes
| no
| yes
| optional
|-
| IPv6 only
| no
| yes
| optional
| yes
|}
All packages providing a DNS server should listen on both protocols by default.


=== Configuration examples ===
=== Configuration examples ===
Line 230: Line 455:
</pre>
</pre>


=== Common DNS issues ===
=== Common misconfigurations ===


==== DNS server not accessible over IPv4 or IPv6 ====
==== No answer to DNS requests ====


Recursive servers need to be accessible over one of the protocols supported by the client machine. Therefore if a network serves both IPv4-only and IPv6-only hosts, the recursive name server must be accessible via both protocols and addresses for both protocols must be advertised via automatic configuration protocols.
Under normal circumstances each request is followed by a DNS reply or ICMP error packet. Failure to deliver any reply at all usually implies a misconfigured firewall dropping all DNS packets for a given protocol.


Authoritative servers need to be accessible over one of the protocols supported by the recursive server. When the condition for recursive servers is met, you can generally assume the recursive server will talk the same protocols the client does. Therefore an authoritative server serving A records should generally be available at least over IPv4 and one serving AAAA records should be available at least over IPv6. An authoritative server serving records for dual-stack servers should therefore be dual-stack as well.
This can be easily simulated using the local firewall, see below.


It is therefore important that all DNS server packages are available on both protocols by default.
<pre>
# iptables -A OUTPUT -p tcp --dport 53 -j DROP                         
# iptables -A OUTPUT -p udp --dport 53 -j DROP
# ip6tables -A OUTPUT -p tcp --dport 53 -j DROP                         
# ip6tables -A OUTPUT -p udp --dport 53 -j DROP
</pre>


==== Lost AAAA request or reply ====
==== No answer to DNS AAAA requests ====


Lost AAAA request is typically caused by the local firewall. Lost AAAA reply may have the same cause but may
This issue is unique because of the selective nature of answer absence. The local resolver typically receives answers for all DNS requests except those asking for AAAA records due to a bug in the authoritative server or any DNS server on the way from the local resolver.
also be the result of buggy recursive DNS server, authoritative DNS server or a firewall issue on the way
between them.


This issue can be simulated by dropping DNS AAAA requests while keeping other requests intact. Unfortunately there seems to be no easy way to do that using conventional firewall. Possible solutions are through delegating the firewall to userspace or using pcap.
This issue is not so easy to simulate as you would have to selectively drop DNS AAAA packets. Possible solutions are through delegating the firewall to userspace or using pcap.


TODO: Provide a way to simulate lost AAAA requests/replies.
TODO: Provide a mechanism to drop DNS AAAA packets, possibly using pcap or broken-dns-proxy.
 
Note: A similar issue with dropped <code>A</code> records is theoretically possible.


Test cases:
Test cases:


* [[../Client_software#Dual-stack_to_dual-stack_with_lost_AAAA_answer|Lost AAAA request or reply]]
* [[../Client_software#Dual-stack_to_dual-stack_with_lost_AAAA_answer|Lost AAAA request or reply]]
Note: A theoretical issue with dropped DNS A records hasn't been observed in the wild.

Latest revision as of 12:25, 15 June 2017

Connectivity

The following table lists basic IPv4 connectivity configurations. There are numerous less common configuration including a combination of public and private addresses and multiple public addresses.

IPv4 connectivity Address Details
Pub Priv LL Lo
Global yes no no yes Host is connected to the Internet using a public address.
Masqueraded no yes no yes Host can use services on the Internet using a masqueraded private address.
Local no yes no yes Host can use local services using a private address.
Link-local no no yes yes Host can use services in the broadcast domain using a link-local address (IPv4LL).
None no no no yes Host has only a loopback address.

The following table lists basic IPv6 connectivity configurations. There are numerous less common configuration including a combination of public and ULA addresses or public or ULA address without a link-local one.

IPv6 connectivity Address Details
Pub ULA LL Lo
Global yes no yes yes Host has at least one link-local address and one global address, the latter being used for global connectivity.
Local no yes yes yes Host has at least one link-local address and one unique local address (ULA) used for local communication.
Link-local no no yes yes Host has only link-local (IPv6LL) and loopback addresses.
None no no no yes Host has only a loopback address.
Disabled no no no no IPv6 has been disabled in the kernel.

The following table lists common or somehow interesting combinations of IPv4 and IPv6 connectivity.

IPv4 \ IPv6 Global Local Link-local None Disabled
Global Dual-stack server IPv4 server
Masqueraded Dual-stack workstation IPv4 workstation IPv4/PPP workstation Disabled IPv6 workstation
Local Private network
Link-local Dual-stack zeroconf IPv4 zeroconf
None IPv6 node IPv6 zeroconf Isolated node

The following table provides details for selected combinations from the previous table.

Scenario Details
Dual-stack server Server available on public IPv4 and IPv6 addresses.
Dual-stack workstation Workstation connecting via both IPv4 and IPv6 Internet services.
IPv4 server Server available on IPv4 with IPv6 limited to link-local and loopback communication.
IPv4 workstation Workstation connecting via IPv4 with IPv6 limited to link-local and loopback communication.
IPv4/PPP workstation Mobile workstation connecting via IPv4 with IPv6 limited to loopback communication as IPv4-only PPP won't assign IPv6LL.
Disabled IPv6 Workstation connecting via IPv4 with IPv6 disabled in the kernel.
IPv6 node Server available on public IPv6 address with IPv4 limited to loopback communication.
Private network Workstation or server on an isolated private network using IPv4 private addresses and IPv6 ULA for local communication.
Dual-stack zeroconf Workstation or appliance in a zero configuration network after a failed DHCPv4 attempt.
IPv6 zeroconf Workstation or appliance in a zero configuration network on a system with IPv4LL disabled.
IPv4 zeroconf Old-school node in a zero configuration network.
Isolated node Disconnected node limited to loopback IPv4 and IPv6 communication.

How to get connectivity for everyday use

Most home and office setups offer masqueraded IPv4 connectivity. In addition they often offer global IPv6 connectivity, often firewalled. Some corporate setups, on the other hand, don't offer connectivity to remote hosts except via a proxy server. Global and unmasqueraded IPv4 and IPv6 connectivity for servers is typically available in datacenters.

IPv4 and IPv6 loopback addresses are assigned automatically at boot time. IPv6 link-local addresses are specific in that they are automatically assigned by the kernel when an interface gets connected to a network on the link layer. On the other hand, IPv4 link-local addresses only serve as a fallback for unsuccessful DHCPv4 attempt.

Note: It looks like link-local IPv4 addresses never get automatically configured in default Fedora 23 Workstation with NetworkManager. This is not necessarily a problem as it is much better to rely on ubiquitous IPv6 link-local addresses nowadays.

IPv6 tunnels

Many networks still only have IPv4 connectivity. You can overcome this limitation by configuring an IPv6 tunnel to a network with IPv6 connectivity. You can use various tunnel technologies to achieve that. There are also services like http://www.tunnelbroker.net/ that offer free tunnelled connectivity with IPv6 subnets.

How to emulate connectivity in a virtual test environment

Using two virtual machines, one can emulate any type of IPv4 and IPv6 connectivity by picking up an address block intended for documentation and testing ( 192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24 and 2001:DB8::/32) and setting up DHCPv4, RA, DHCPv6 and DNS services on one of the virtual machines, while using the other as a client node.

Router with automatic configuration

Server always needs to have a valid static configuration. For simple testing, you can configure the server temporarily using iproute package.

interface=eth1

ip address add 192.0.2.1/24 dev $interface
ip address add 2001:db8::1/64 dev $interface
ip link set $interface up

The easiest way to provide a testing IPv4 and IPv6 configuration server together with a local DNS server is using dnsmasq. The command bellow can be easily transformed into permanent configuration in dnsmasq.conf.

dnsmasq -d \
    --dhcp-range=lan,192.0.2.100,192.0.2.200,10m \
    --enable-ra --dhcp-range=lan,2001:db8::1:0,2001:db8::2:0,10m

Automatically configured node

A proper workstation installation will get configured automatically using NetworkManager. A server can do exactly the same especially if the router is configured with static address assignments via DHCP. Or the server can be configured manually.

On the other hand you can test the server configuration using dhcpcd command from the dhcpcd package. Use the following command. If you leave out the -T option, dhcpcd would configure kernel and /etc/resolv.conf using the information from the server.

dhcpcd -d -T eth0

Note: There are other non-default configuration tools including systemd-networkd and connman but those are out of scope of this document for now.

How to test network scenarios using Linux network namespaces

A pair of veth devices connecting two namespaces

clientns="test-client"
clientif=$clientns
serverns="test-server"
serverif="$serverns


# setup the network namespaces
ip netns add $serverns
ip netns add $clientns

# add veth device
ip link add dev $serverif type veth peer name $clientif

# add the veth device endpoints to the appropriate network namespace
ip link set $clientif netns $clientns
ip link set $serverif netns $serverns

# enable the loopback interfaces
ip netns exec $clientns ip link set lo up
ip netns exec $serverns ip link set lo up

Router with automatic configuration

# configure the static ip addresses on the interface where dnsmasq will be run
ip netns exec $serverns ip address add 192.0.2.1/24 dev $serverif
ip netns exec $serverns ip address add 2001:db8::1/64 dev $serverif

# activate the interface
ip netns exec $serverns ip link set $serverif up

# start dnsmasq as the DHCP server and DNS resolver
ip netns exec $serverns \
    dnsmasq -d \
    --dhcp-range=lan,192.0.2.100,192.0.2.200,10m \
    --enable-ra --dhcp-range=lan,2001:db8::1:0,2001:db8::2:0,10m

Automatically configured node

ip netns exec test-node \
    dhcpcd -d -T eth0
ip netns exec test-node \
    ping router.example.net

Notes on connectivity checks

Network application behavior may change based on various connectivity checks. Some of them work with IP addresses, some with routes and some are even based on attempting connections. All of the mentioned types of checks are currently present in glibc name resolution code, so you need to be careful about subtle differences in configuration.

DNS configuration

A client connects to a service using IPv4 or IPv6 addresses. In many cases the client retrieves a domain name from user configuration or user input and uses that domain name to query DNS for the IP addresses. The most universal translation of a service domain to an IP address is through a combination of SRV and A/AAAA records. First the service is translated to a set of host names using the SRV record, then each host name is translated to a set of IPv4/IPv6 addresses using the A/AAAA records. In many cases indirection using SRV records is not supported and the host name must also be used as the service names. In some cases a different record (MX, NS, ...) is used for the indirection.

The intermediate records (SRV, MX, NS, ...) are protocol neutral and vary by service. At least one A and at least one AAAA service is needed for proper operation of a dual-stack service. IPv4 only and IPv6 only services only have one or more A or AAAA record respectively. Configuration with multiple A and AAAA records (especially in combination with multiple SRV or other intermediate records) are useful for DNS based failover.

Key requirements:

  • Domain name of the service is known (from user configuration or input)
  • Optional: Authoritative DNS zone provides a list of host names for the service domain name
  • Authoritative DNS zone provides correct A/AAAA records for the host domain name
  • Service is available on addresses in A/AAAA records
  • Recursive DNS server must be able to retrieve he above records
  • Local host must be able tu query the DNS server and connect to at least some of the addresses

Possible reactions to DNS requests:

Reply type DNS rcode Meaning Details
DNS NOERROR Data Reply with one or more records of the requested type.
DNS NXDOMAIN No such domain Reply saying that there are no records at all for the domain name.
DNS NODATA No data Reply saying that there are records for the domain name but not of the requested type.
ICMP Destination unreachable The server isn't available for communication.
No reply at all The server failed to reply or the reply was dropped on the firewall.

The test environment often requires a local recursive and authoritative DNS server. In most cases a dnsmasq instance or any other forwarding nameserver with some authoritative capabilities will do the job. For testing, domains can be chosen under example.net and example.com subtrees.

DNS server accessibility

Domain name servers serve different roles. Authoritative servers provide zone data configured by service administrators. Recursive servers retrieve zone data from authoritative servers on behalf of the clients.

Recursive servers are part of the client infrastructure. In a dual-stack network they need to be be available on both IPv4 and IPv6 to also serve single-stack clients using any of the protocols and addresses for both protocols should be announced using automatic configuration protocols. In setups with additinal forwarding DNS servers that do not perform full recursion this applies to those DNS servers that actually serve clients.

The following table contains common combinations of dual-stack, IPv4 only and IPv6 only network network clients and the respective requirements on recursive DNS service availability.

Client IPv4/IPv6 support RDNSS requirements
Dual-stack IPv4 only IPv6 only IPv4 IPv6
yes/no yes yes yes yes
yes/no yes no yes optional
yes/no no yes optional yes
yes no no at least one of them

Authoritative servers are part of the service infrastructure. For dual-stack services they need to be available on both protocols in order to serve single-stack recursive DNS servers. For single-stack services they need to be available on the respective protocol.

Service connectivity DNS records Authoritative DNS
A AAAA IPv4 IPv6
Dual-stack yes yes yes yes
IPv4 only yes no yes optional
IPv6 only no yes optional yes

All packages providing a DNS server should listen on both protocols by default.

Configuration examples

Dual-stack server

Using dnsmasq, /etc/dnsmasq.conf follows:

address=/server.example.net/192.0.2.1
address=/server.example.net/2001:DB8::1

Testing with netresolve:

$ netresolve --node server.example.net --protocol raw

Dual-stack server with SRV record

Using dnsmasq, /etc/dnsmasq.conf follows:

srv-host=_http._tcp.example.net,server.example.net,80,10,10
address=/server.example.net/192.0.2.1
address=/server.example.net/2001:DB8::1

Testing with netresolve:

$ netresolve --srv --node example.net --service http --protocol tcp

Common misconfigurations

No answer to DNS requests

Under normal circumstances each request is followed by a DNS reply or ICMP error packet. Failure to deliver any reply at all usually implies a misconfigured firewall dropping all DNS packets for a given protocol.

This can be easily simulated using the local firewall, see below.

# iptables -A OUTPUT -p tcp --dport 53 -j DROP                           
# iptables -A OUTPUT -p udp --dport 53 -j DROP
# ip6tables -A OUTPUT -p tcp --dport 53 -j DROP                           
# ip6tables -A OUTPUT -p udp --dport 53 -j DROP

No answer to DNS AAAA requests

This issue is unique because of the selective nature of answer absence. The local resolver typically receives answers for all DNS requests except those asking for AAAA records due to a bug in the authoritative server or any DNS server on the way from the local resolver.

This issue is not so easy to simulate as you would have to selectively drop DNS AAAA packets. Possible solutions are through delegating the firewall to userspace or using pcap.

TODO: Provide a mechanism to drop DNS AAAA packets, possibly using pcap or broken-dns-proxy.

Test cases:

Note: A theoretical issue with dropped DNS A records hasn't been observed in the wild.