User:Crantila/FSC/Real Time

This page is a draft only
It is still under construction and content may change. Do not rely on the information on this page.

Real-Time and Low Latency

It is perhaps a common perception that computers can compute things instantaneously. Anybody who has ever waited for a web page to load has first-hand experience that this is not the case: computers take time to do things, even if the amount of time is often imperceptible to human observers. Moreover, a computer doing one thing can seem like it's acting nearly instantaneously, but a computer doing fifteen things will have a more difficult time keeping up appearances.

When computer audio specialists talk about a computer acting in real-time, they mean that it is acting with only an imperceptible delay. A computer cannot act on something instantaneously, and the amount of waiting time between an input and its output is called latency. In order for the delay between input and output to be perceived as non-existant (in other words, for a computer to "react in real-time,") the latency must be low.

For periodic tasks, like processing audio (which has a consistently recurring amount of data per second), low latency is desirable, but consistent latency is usually more important. Think of it like this: years ago in North America, milk was delivered to homes by a dedicated delivery person. Imagine if the milk delivery person had a medium-latency, but consistent schedule, returning every seven days. You would be able to plan for how much milk to buy, and to limit your intake so that you don't run out too soon. Now imagine if the milk delivery person had a low-latency, but inconsistent schedule, returning every one to four days. You would never be sure how much milk to buy, and you wouldn't know how to limit yourself. Sometimes there would be too much milk, and sometimes you would run out. Audio-processing and synthesis software behaves in a similar way: if it has a consistent amount of latency, it can plan accordingly. If it has an inconsistent amount of latency - whether large or small - there will sometimes be too much data, and sometimes not enough. If your application runs out of audio data, there will be noise or silence in the audio signal - both bad things.

Relatively low latency is still important, so that your computer reacts imperceptibly quickly to what's going on. The point is that the difference between an 8ms target latency and a 16ms target latency is almost certainly imperceptible to humans, but the higher latency may help your computer to be more consistent - and that's more important.

Processor Scheduling

If you've ever opened the "System Monitor" application, you will probably have noticed that there are a lot of "processes" running all the time. Some of these processes need the processor, and some of them are just waiting around for something to happen. To help increase the number of processes that can run at the same time, many modern CPUs have more than one "core," which allows for more processes to be evaluated at the same time. Even with these improvements, there are usually more processes than available cores: my computer right now has 196 processes and only three cores. There has to be a way of decided which process gets to run and when, and this task is left to the operating system.

In GNU/Linux systems like Fedora Linux, the core of the operating system (called the kernel) is responsible for deciding which process gets to execute at what time. This responsibility is called "scheduling." Scheduling access to the processor is called, processor scheduling. The kernel also manages scheduling for a number of other things, like memory access, video hardware access, audio hardware access, hard drive access, and so on. The algorithm (procedure) used for each of these scheduling tasks is different for each, and can be changed depending on the user's needs and the specific hardware being used. In a hard drive, for example, it makes sense to consider the physical location of data on a disk before deciding which process gets to read first. For a processor this is irrelevant, but there are many other things to consider.

There are a number of scheduling algorithms that are available with the standard Linux kernel, and for most uses, a "fair queueing" system is appropriate. This helps to ensure that all processes get an equal amount of time with the processor, and it's unacceptable for audio work. If you're recording a live concert, and the "PackageKit" update manager starts, you don't care if PackageKit gets a fair share of processing time - it's more important that the audio is recorded as accurately as possible. For that matter, if you're recording a live concert, and your computer isn't fast enough to update the monitor, keyboard, and mouse position while providing uninterrupted, high-quality audio, you want the audio instead of the monitor, keyboard, and mouse. After all, once you've missed even the smallest portion of audio, it's gone for good!

The Real-Time Linux Kernel

There is a "real-time patch" for the Linux kernel which enables the processor to unfairly schedule certain processes that ask for higher priority. Although the term "patch" may make it seem like this is just a partial solution, it really refers to the fact that the programming code used to enable this kind of unfair scheduling is not included in standard kernels; the standard kernel code must have this code "patched" into it.

The default behaviour of a real-time kernel is still to use the "fair queueing" system by default. This is good, because most processes don't need to have consistently low latencies. Only specific processes are designed to request high-priority scheduling. Each process is given (or asks for) a priority number, and the real-time kernel will always give processing time to the process with the highest priority number, even if that process uses up all of the available processing time. This puts regular applications at a disadvantage: when a high-priority process is running, the rest of the system may be unable to function properly. In extreme (and very rare!) cases, a real-time process can encounter an error, use up all the processing time, and disallow any other process from running - effectively locking you out of your computer. Security measures have been taken to help ensure this doesn't happen, but as with anything, there is no guarantee. If you use a real-time kernel, you are exposing yourself to a slightly higher risk of system crashes.

A real-time kernel should not be used on a computer that acts as a server, for these reasons.

Hard and Soft Real-Time

Finally, there are two different kinds of real-time scheduling. The Linux kernel, even at its most extreme, uses only soft real-time. This means that, while processor and other scheduling algorithms may be optimized to give preference to higher-priority processes, no absolute guarantee of performance can be made. A real-time kernel helps to greatly reduce the chance of an audio process running out of data, but sometimes it can still happen.

A hard real-time computer is designed for specialized purposes, where even the smallest amount of latency can make the difference between life and death. These systems are implemented in hardware as well as software. Example uses include triggering airbag deployment in automobile crashes, and monitoring the heart rate of a patient during an operation. These computers are not particularly multi-functional, which is part of their means to accomplishing a guaranteed low latency.

Getting a Real-Time Kernel in Fedora

In Fedora Linux, the real-time kernel is provided by the Planet CCRMA at Home software repositories. Along with the warnings in the Planet CCRMA section, here is one more to consider: the real-time kernel is used by fewer people than the standard kernel, so it is less well-tested. The changes of something going wrong are relatively low, but be aware that using a real-time kernel increases the level of risk. Always leave a non-real-time option available, in case the real-time kernel stops working.

You can install the real-time kernel, along with other system optimizations, by following these instructions:

1. Install the Planet CCRMA at Home repositories by following these instructions.
2. Run the following command in a terminal:

   su -c 'yum install planetccrma-core'

   Note that this is a meta-package, which does not install anything by itself, but causes a number of other packages to be installed, which will themselves perform the desired installation and optimization.
3. Shut down and reboot your computer, to test the new kernel. If you decided to modify your GRUB configuration, be sure that you leave a non-real-time kernel available for use.