Setting up netconsole
Assume we have two computers on the same network.
The machine with a problem (the debugee):
IP address: 192.168.1.1
And a second machine (the debugger):
IP address: 192.168.1.2 MAC address of eth0: 00:11:22:33:44:55
The debugger
On a different machine than the one you are seeing the problem, you need to run nc -l -u 6666
This will listen on udp port 6666 for any messages.
The debugee
On the machine that you're trying to debug a problem, run (as root) modprobe netconsole netconsole=4444@192.168.1.1/eth0,6666@192.168.1.2/00:11:22:33:44:55
If you replace the IP/MAC addresses with those relevant to your machines/network in the command above, you should see messages appearing on the debugging machine.
For more information on netconsole, see http://www.mjmwired.net/kernel/Documentation/networking/netconsole.txt
Comment: There should be some explicit statement about whether iptables and/or the firewall on the debugger needs to be changed temporarily, etc. Also, is there some other way to use netconsole so that it is active before you can invoke modprobe from a shell, such as during boot itself?
logging levels
You may want to raise the level of printk's that go to the console.
To do this when logged in:
dmesg -n 8
netconsole at boot
Raise the level of console by default logging by adding this line to /etc/sysctl.conf:
kernel.printk = 8 4 1 7
Add your host info to netconsole:
$ cat /etc/modprobe.d/netconsole.conf options netconsole