Ok, so you have installed the real time kernel form Planet CCRMA but you feel that something is still compromising your low latency objectives. Well, this is touchy subject and it does have many variables, but with some research and some useful tools, you will gain the knowledge to have your Fedora machine running with acceptable latencies.
Tools
There are a two tools that I use to further fine tune my system for real time audio use.
The first one is the real time configuration quick scan. You can download it here: http://code.google.com/p/realtimeconfigquickscan/ [1]
Follow the instructions on the site.
If you want to run a command line version of this helpful script, simply navigate to the /realtimeconfigquickscan directory and type "./realTimeConfigQuickScan.pl" and hit enter.
I recommend running the command line version as the GUI version is still experimental.