MULTI for Embedded Linux

Traditionally, Linux engineers have had to rely on debugging tools that offered only limited visibility and usefulness. GDB—and its related front-ends such as Eclipse—are cumbersome to set up, difficult to use, and not always trustworthy. As a result, developers often resort to the hit-and-miss effectiveness of printf or printk commands. Unfortunately, these are problematic as well and provide only a very small window into complex system execution.

With MULTI for Linux, embedded engineers can now visualize and debug all aspects of an embedded Linux system—including the Linux kernel, kernel threads, interrupt services routines, as well as user mode threads and processes—faster and more easily than ever before.

MULTI for Linux offers a range of sophisticated debugging tools:

• comprehensive Linux kernel visualization—including shared memory regions, semaphores, message queues, and kernel modules
• support for SMP multicore debugging
• debug from kernel through user space with one debugger
• debug native Linux and virtualized Linux (on INTEGRITY Multivisor virtualization technology)
• debug loadable kernel modules and device drivers
• easily debug process startup code
• set any thread-specific breakpoints in kernel code
• debug shared objects in virtual address spaces
• handle breakpoints with minimum performance cost
• debug multi-threaded interactions across processes

Application debugging with run-mode

Run-mode debugging of user-space applications gives programmers accurate and comprehensive visibility into real-world system behavior. To start debugging with MULTI, you simply load Linux and run your program. With just a few clicks, you can easily step through and browse even very large blocks of code with no lag time to:

• view variables, data structures, and the call stack
• view and modify memory locations
• set hardware breakpoints
• detect and load symbols for shared libraries
• debug code right after a fork

Using run-mode debugging, Linux engineers can zero in on a specific area while the rest of the system keeps executing without disruption. You can halt, run, and set breakpoints on one specific thread—even if it uses code common to others—while the remaining threads continue to run.

The system information listed in the Linux Kernel Object Information window provides a detailed view of the complex internals of the Linux kernel as well as applications running on the system. Click for larger view

Debugging on multicore processors

Green Hills Software was one of the first companies to offer a probe and debugger solution for multicore systems. MULTI for Linux includes features that make debugging multicore systems as straightforward as single core systems.

Key among these features is synchronous run control, which halts all cores as a unit when any core encounters a debugging condition. For instance, when a core hits a breakpoint the target list clearly shows:

• which core(s) have hit the breakpoint
• what thread was executing on the core at the time
• what the other cores in the system were doing

Debugging virtualized Linux systems

Debugging is also straightforward on systems where Linux runs on a virtual machine (using INTEGRITY Multivisor). Traditionally, these kinds of systems would require multiple debuggers—a JTAG device alongside a console window to view print statements and GDB for the Linux machine.

But MULTI can debug everything from low-level INTEGRITY device drivers to Linux kernel drivers—virtualized or not—to applications running in Linux. No matter where your bug is, you will have full control and visibility to debug it.

Debugging kernel modules

Debugging in the Linux kernel space can be complicated—usually requiring special commands or GDB workarounds.

But MULTI’s comprehensive OS awareness enables straightforward debugging across the user space and into the kernel. Working in freeze-mode, MULTI can automatically determine which kernel modules are loaded to allow users to seamlessly debug device drivers and their associated user space applications.

MULTI for Linux can provide debug visibility throughout systems that combine Linux and INTEGRITY operating systems running in both native and virtualized modes.