Debugging¶

27 Aug 2022

Advanced users may be interested in running the analyzer under a debugger. This is usually the fastest way to track down mysterious unexpected behavior or program crashes, certainly faster than adding print statements to the code and going through repeated edit-compile-run cycles. Jump to Debug Sessions below to see an example.

Debug Build¶

By default, the analyzer is built with debug symbols (CMake build type RelWithDebInfo) so that one can run the debugger right away without rebulding anything. That said, because of compiler optimizations, the debugger will sometimes appear to show the wrong program flow, some variables may not be available for inspection, etc. For the best debugging results, it may therefore be helpful to create a dedicated debug build. To do so, build the analyzer with the CMake build type "Debug":

% cmake -B build-debug -S . -DCMAKE_BUILD_TYPE=Debug
% cmake --build build-debug -j

You might also want to link against a debug version of ROOT or install an appropriate debuginfo package (e.g. root-debuginfo on RedHat systems using EPEL). A true debug version of ROOT requires building ROOT from source, again using CMAKE_BUILD_TYPE=Debug, which is beyond the scope of this document.

Also, on RedHat Linux be sure to install glibc-debuginfo, glibc-debuginfo-common, and libstdc++-debuginfo to get debug support for STL classes and other standard library features. One Debian/Ubuntu systems, the corresponding packages are libc6-dbg and libstdc++6-N-dbg, where N should be replaced with the version of the gcc in use. Package names and availability may vary between different Linux versions. These steps are not necessary on macOS, where the relevant bundles come with the required symbol tables included.

Debug Sessions¶

The recommended debugger is gdb on Linux and lldb on macOS. To avoid problems with macOS's System Integrity Protection when using lldb, it may be helpful to not use /usr/bin/lldb directly, but instead create an alias to the command-line tools version, which isn't a specially-protected executable:

% alias lldb='/Library/Developer/CommandLineTools/usr/bin/lldb'

The analyzer may be run directly from the build location under the debugger of your choice:

% ./build-debug/apps/analyzer

Once the analyzer is running, open a second terminal and find out the PID of this analyzer process. Then "attach" the debugger, which will stop the analzyer and allow you to configure the session, e.g. to set break points, etc. For example, on Linux using gdb:

% ps a | grep analyzer
13643 pts/3    S+     0:00 ./build-debug/apps/analyzer
% gdb
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-120.el7
...
(gdb) attach 13643
...
(gdb) break THaRun::THaRun
Breakpoint 1 at 0x7fab342a10a7: THaRun::THaRun. (3 locations)
(gdb) continue
Continuing.

This sets a breakpoint for all versions of the THaRun constructor. The continue command (shorthand c) resumes execution of the program.

Go back to the analyzer prompt and create a THaRun object. The process will stop and print information (in the debugger window), allowing you to inspect data, step through the code, etc.:

analyzer [0] r = new THaRun("/data/raw/run_7211.evio.0")

In the debugger window, you'll see

Breakpoint 1, THaRun::THaRun (this=0x503bb20,
    fname=0x7fab188f002c "/data/raw/run_7211.evio.0",
    description=0x7fab188f004f "")
    at /home/ole/Develop/analyzer/Podd/THaRun.cxx:39
39      fSegment(-1), fStream(-1)
(gdb) p fname
$1 = 0x7fab188f002c "/data/raw/run_7211.evio.0" 
(gdb)

Here, we printed (shorthand p) the contents of the variable fname. To end the session, type q in the debugger window.

The process is very similar under macOS with lldb, although the syntax of some lldb commands differs from that of gdb. You can find side-by-side comparisons of gdb/lldb commands online. Obviously, this only scratches the surface of what is possible with a debugger. If you are unfamiliar with gdb or lldb, there are plenty of tutorials available online that should get you started quickly.