We are experiencing kernel panics in the field for our MIPS based embedded devices. How can I log the kernel panic trace in the MTD partition? Do we have to write the trace only into MTD or is it possible to write over the NFS? Can anyone explain how to get useful kernel traces after the panic for the remote boxes.
Mips Linux: Logging Kernel Panic into mtd partition
5.5k views Asked by user4117880 At
2
There are 2 answers
0
On
Thanks so much for the detailed steps. Everything seems to be fine, except the device name. When I give the device name as "/dev/oops" mtdoops is not attached to any device in the boot message. After a level of debugging , I found that the device name shall be just "oops" or "Partition number" i.e (if your oops partition is mtd9 just give "9" as partition number). After changing it , it started working. I am able to see the logs using sp-oops-extract.
You can turn on mtdoops module in the kernel and log the kernel panic traces on to mtd partition. I don't think we can write the panic trace over NFS. However, you may want to explore about ramoops.
The following are the steps to configure the kernel to capture the kernel oops on to the mtd flash. Capturing the stack trace after the kernel panic is invaluable to debug the kernel problems particularly the ones happening in the field. During mtdoops module init, the mtd partition is turned into a circular buffer and erased beforehand.
The kernel flag, CONFIG_MTD_OOPS, configures the kernel to write the oops stack trace on to the MTD partition. This MTD dev partition information can be hard coded inside mtdoops module or can be specified dynamically. This component can be built as part of kernel or as an separate module. Before building the kernel, you need to make sure that your mtd device has registered a panic_write handler. Remember, normal mtd write handler won't be enough as we have to write to mtd memory after kernel panic. Please run this patch if the mtd device doesn't have its own a panic write handler.
When built as part of the kernel, CONFIG_MTD_OOPS=y, the mtdoops module needs to be patched with the flash partition information (mtddev).
While building it as a module, CONFIG_MTD_OOPS=m, the flash partition information is provided dynamically during module installation (insmod).
In addition to enable the MTP OOPS flags, do configure, CONFIG_MAGIC_SYSRQ, to induce the panic and to test this functionality.