StarPU Handbook
 All Data Structures Files Functions Variables Typedefs Enumerations Enumerator Macros Groups Pages
Debugging Tools

StarPU provides several tools to help debugging applications. Execution traces can be generated and displayed graphically, see Generating Traces With FxT.

TroubleShooting In General

Generally-speaking, if you have troubles, pass --enable-debug to ./configure to enable some checks which impact performance, but will catch common issues, possibly earlier than the actual problem you are observing, which may just be a consequence of a bug that happened earlier. Also, make sure not to have the --enable-fast option which drops very useful catchup assertions. If your program is valgrind-safe, you can use it, see Using Other Debugging Tools.

Depending on your toolchain, it might happen that you get undefined reference to `__stack_chk_guard' errors. In that case, use the –disable-fstack-protector-all option to avoid the issue.

Then, if your program crashes with an assertion error, a segfault, etc. you can send us the result of

thread apply all bt

run in gdb at the point of the crash.

In case your program just hangs, but it may also be useful in case of a crash too, it helps to source gdbinit as described in the next section to be able to run and send us the output of the following commands:

starpu-workers
starpu-tasks
starpu-print-requests
starpu-print-prequests
starpu-print-frrequests
starpu-print-irrequests

To give us an idea of what is happening within StarPU. If the outputs are not too long, you can even run

starpu-all-tasks
starpu-print-all-tasks
starpu-print-datas-summary
starpu-print-datas

Using The Gdb Debugger

Some gdb helpers are provided to show the whole StarPU state:

(gdb) source tools/gdbinit
(gdb) help starpu

For instance,

  • one can print all tasks with starpu-print-all-tasks,
  • print all datas with starpu-print-datas,
  • print all pending data transfers with starpu-print-prequests, starpu-print-requests, starpu-print-frequests, starpu-print-irequests,
  • print pending MPI requests with starpu-mpi-print-detached-requests

Some functions can only work if --enable-debug was passed to ./configure (because they impact performance)

Using Other Debugging Tools

Valgrind can be used on StarPU: valgrind.h just needs to be found at ./configure time, to tell valgrind about some known false positives and disable host memory pinning. Other known false positives can be suppressed by giving the suppression files in tools/valgrind/*.suppr to valgrind's –suppressions option.

The environment variable STARPU_DISABLE_KERNELS can also be set to 1 to make StarPU does everything (schedule tasks, transfer memory, etc.) except actually calling the application-provided kernel functions, i.e. the computation will not happen. This permits to quickly check that the task scheme is working properly.

Using The Temanejo Task Debugger

StarPU can connect to Temanejo >= 1.0rc2 (see http://www.hlrs.de/temanejo), to permit nice visual task debugging. To do so, build Temanejo's libayudame.so, install Ayudame.h to e.g. /usr/local/include, apply the tools/patch-ayudame to it to fix C build, re-./configure, make sure that it found it, rebuild StarPU. Run the Temanejo GUI, give it the path to your application, any options you want to pass it, the path to libayudame.so.

It permits to visualize the task graph, add breakpoints, continue execution task-by-task, and run gdb on a given task, etc.

temanejo.png

Make sure to specify at least the same number of CPUs in the dialog box as your machine has, otherwise an error will happen during execution. Future versions of Temanejo should be able to tell StarPU the number of CPUs to use.

Tag numbers have to be below 4000000000000000000ULL to be usable for Temanejo (so as to distinguish them from tasks).