From Fedora Project Wiki

Network Manager Test Suites

Summary

The NetworkManager project has recently recieved a bunch of new features and other features are under development. For Fedora, NetworkManager is the network configuration solution of choice. The goal of this feature is to prepare several test suites for NetworkManager and related projects and add them to the upstream project.

Owner

Current status

  • Targeted release: Fedora 20
  • Last updated: 2013-01-21
  • Percentage of completion: 20%

Detailed description

There will be several test suites to fulfill specific goals. Together they will cover not only NetworkManager itself but also some of its dependencies.

We are submitting this Feature because:

  • NetworkManager is one of the most important parts of the networking stack
  • We believe that reliability is as important as the features themselves
  • It depends on a Fedora project called LNST
  • The test results uncover bugs in other packages including kernel and libnl
  • We want to attract community members to contribute more tests

Underlying platform test suite

Routines to access the underlying platform is being be centralized into a single NetworkManager module called nm-platform. The purpose of this test suite is to check the nm-platform module, the underlying libnl library and the kernel. It requires root access to the machine where it is used.

Core behavior test suite

The nm-platform module offers a fake implementation of the underlying system behavior. This can be used to test the behavior of core NetworkManager code. The purpose of the resulting test suite is to carefully test the inner workings of NetworkManager. This test suite won't have any specific requirements and will run as 'make check' test even with an ordinary user account.

LNST-based virtual network test suite

The LNST project can be used to run larger scale integration tests for specific network configurations. The purpose of this test suite is to test the real-world use cases and configurations. It will be the only test suite that will go as far as testing the actual connectivity. It will require LNST testing framework and a virtualization solution. Alternatively, a network of physical hosts can be used.

Benefit to Fedora

Most Fedora installations are being used in networking environments with NetworkManager handling the configuration. Bugs and regressions always affect the usability and reputation of Fedora. We believe that the testsuites will help to avoid it as well as save developers' time spent on manual tests. Fedora will benefit from being able offer reliability and avoid surprising and buggy behaviour.

Scope

We want to include all the tests into NetworkManager's upstream git master. Even though NetworkManager is the main affected project here, various tests will affect various project NetworkManager relies on.

We expect the following projects to recieve feedback from the testsuites:

  • Linux kernel
  • libnl
  • NetworkManager
  • dhclient
  • dnsmasq

(other project will be added during the work)

How to test

The scope of this feature is the testing itself. To test this feature, run all the testsuites. Some of them are suitable for testing specific hardware and software combinations, especially the “underlying platform test suite”.

User experience

By improving reliability of the network stack, the user experience will improve on all levels.

Dependencies

This feature requires a stable release of LNST suitable for NetworkManager testing. One of the feature owners (Radek) is working on LNST and will help us in this area.

Contingency plan

If test suites are not ready for the target release, NetworkManager will be used just as it was used before. Even partial completion of the feature helps.

Documentation

The test suites are mostly self-documenting but during the work we will try to improve both information on the Fedora Wiki and the upstream documentation.

Release notes

NetworkManager now comes with test suites that greatly improve its reliability and usability in various environments.

Comments and discussions

See Talk:Features/NetworkManagerTestSuites