StarPU Handbook
Online Performance Tools

# On-line Performance Feedback

## Enabling On-line Performance Monitoring

In order to enable online performance monitoring, the application can call starpu_profiling_status_set() with the parameter STARPU_PROFILING_ENABLE. It is possible to detect whether monitoring is already enabled or not by calling starpu_profiling_status_get(). Enabling monitoring also reinitialize all previously collected feedback. The environment variable STARPU_PROFILING can also be set to 1 to achieve the same effect. The function starpu_profiling_init() can also be called during the execution to reinitialize performance counters and to start the profiling if the environment variable STARPU_PROFILING is set to 1.

Likewise, performance monitoring is stopped by calling starpu_profiling_status_set() with the parameter STARPU_PROFILING_DISABLE. Note that this does not reset the performance counters so that the application may consult them later on.

More details about the performance monitoring API are available in Profiling.

If profiling is enabled, a pointer to a structure starpu_profiling_task_info is put in the field starpu_task::profiling_info when a task terminates. This structure is automatically destroyed when the task structure is destroyed, either automatically or by calling starpu_task_destroy().

The structure starpu_profiling_task_info indicates the date when the task was submitted (starpu_profiling_task_info::submit_time), started (starpu_profiling_task_info::start_time), and terminated (starpu_profiling_task_info::end_time), relative to the initialization of StarPU with starpu_init(). It also specifies the identifier of the worker that has executed the task (starpu_profiling_task_info::workerid). These date are stored as timespec structures which the user may convert into micro-seconds using the helper function starpu_timing_timespec_to_us().

It it worth noting that the application may directly access this structure from the callback executed at the end of the task. The structure starpu_task associated to the callback currently being executed is indeed accessible with the function starpu_task_get_current().

## Per-codelet Feedback

The field starpu_codelet::per_worker_stats is an array of counters. The i-th entry of the array is incremented every time a task implementing the codelet is executed on the i-th worker. This array is not reinitialized when profiling is enabled or disabled.

## Per-worker Feedback

The second argument returned by the function starpu_profiling_worker_get_info() is a structure starpu_profiling_worker_info that gives statistics about the specified worker. This structure specifies when StarPU started collecting profiling information for that worker (starpu_profiling_worker_info::start_time), the duration of the profiling measurement interval (starpu_profiling_worker_info::total_time), the time spent executing kernels (starpu_profiling_worker_info::executing_time), the time spent sleeping because there is no task to execute at all (starpu_profiling_worker_info::sleeping_time), and the number of tasks that were executed while profiling was enabled. These values give an estimation of the proportion of time spent do real work, and the time spent either sleeping because there are not enough executable tasks or simply wasted in pure StarPU overhead.

Calling starpu_profiling_worker_get_info() resets the profiling information associated to a worker.

To easily display all this information, the environment variable STARPU_WORKER_STATS can be set to 1 (in addition to setting STARPU_PROFILING to 1). A summary will then be displayed at program termination:

Worker stats:
CUDA 0.0 (4.7 GiB)
total: 1574.82 ms executing: 1510.72 ms sleeping: 0.00 ms overhead 64.10 ms
325.217970 GFlop/s

CPU 0
total: 1574.82 ms executing: 1364.81 ms sleeping: 0.00 ms overhead 210.01 ms
7.512057 GFlop/s

CPU 1
total: 1574.82 ms executing: 1500.13 ms sleeping: 0.00 ms overhead 74.69 ms
6.675853 GFlop/s

CPU 2
total: 1574.82 ms executing: 1553.12 ms sleeping: 0.00 ms overhead 21.70 ms
7.152886 GFlop/s


The number of GFlops is available because the starpu_task::flops field of the tasks were filled (or STARPU_FLOPS used in starpu_task_insert()).

When an FxT trace is generated (see Generating Traces With FxT), it is also possible to use the tool starpu_workers_activity (see Monitoring Activity) to generate a graphic showing the evolution of these values during the time, for the different workers.

## Bus-related Feedback

The bus speed measured by StarPU can be displayed by using the tool starpu_machine_display, for instance:

StarPU has found:
3 CUDA devices
CUDA 0 (Tesla C2050 02:00.0)
CUDA 1 (Tesla C2050 03:00.0)
CUDA 2 (Tesla C2050 84:00.0)
from    to RAM          to CUDA 0       to CUDA 1       to CUDA 2
RAM     0.000000        5176.530428     5176.492994     5191.710722
CUDA 0  4523.732446     0.000000        2414.074751     2417.379201
CUDA 1  4523.718152     2414.078822     0.000000        2417.375119
CUDA 2  4534.229519     2417.069025     2417.060863     0.000000


Statistics about the data transfers which were performed and temporal average of bandwidth usage can be obtained by setting the environment variable STARPU_BUS_STATS to 1; a summary will then be displayed at program termination:

Data transfer stats:
RAM 0 -> CUDA 0	319.92 MB	213.10 MB/s	(transfers : 91 - avg 3.52 MB)
CUDA 0 -> RAM 0	214.45 MB	142.85 MB/s	(transfers : 61 - avg 3.52 MB)
RAM 0 -> CUDA 1	302.34 MB	201.39 MB/s	(transfers : 86 - avg 3.52 MB)
CUDA 1 -> RAM 0	133.59 MB	88.99 MB/s	(transfers : 38 - avg 3.52 MB)
CUDA 0 -> CUDA 1	144.14 MB	96.01 MB/s	(transfers : 41 - avg 3.52 MB)
CUDA 1 -> CUDA 0	130.08 MB	86.64 MB/s	(transfers : 37 - avg 3.52 MB)
RAM 0 -> CUDA 2	312.89 MB	208.42 MB/s	(transfers : 89 - avg 3.52 MB)
CUDA 2 -> RAM 0	133.59 MB	88.99 MB/s	(transfers : 38 - avg 3.52 MB)
CUDA 0 -> CUDA 2	151.17 MB	100.69 MB/s	(transfers : 43 - avg 3.52 MB)
CUDA 2 -> CUDA 0	105.47 MB	70.25 MB/s	(transfers : 30 - avg 3.52 MB)
CUDA 1 -> CUDA 2	175.78 MB	117.09 MB/s	(transfers : 50 - avg 3.52 MB)
CUDA 2 -> CUDA 1	203.91 MB	135.82 MB/s	(transfers : 58 - avg 3.52 MB)
Total transfers: 2.27 GB


## MPI-related Feedback

Statistics about the data transfers which were performed over MPI can be obtained by setting the environment variable STARPU_COMM_STATS to 1; a summary will then be displayed at program termination:

[starpu_comm_stats][0] TOTAL:	4.000000 GB	4.000000 GB
[starpu_comm_stats][0->1]	4.000000 GB	4.000000 GB
[starpu_comm_stats][1] TOTAL:	8.000000 GB	8.000000 GB
[starpu_comm_stats][1->0]	8.000000 GB	8.000000 GB


## StarPU-Top Interface

StarPU-Top is an interface which remotely displays the on-line state of a StarPU application and permits the user to change parameters on the fly.

Variables to be monitored can be registered by calling the functions starpu_top_add_data_boolean(), starpu_top_add_data_integer(), starpu_top_add_data_float(), e.g.:

starpu_top_data *data = starpu_top_add_data_integer("mynum", 0, 100, 1);

The application should then call starpu_top_init_and_wait() to give its name and wait for StarPU-Top to get a start request from the user. The name is used by StarPU-Top to quickly reload a previously-saved layout of parameter display.

starpu_top_init_and_wait("the application");

The new values can then be provided thanks to starpu_top_update_data_boolean(), starpu_top_update_data_integer(), starpu_top_update_data_float(), e.g.:

Updateable parameters can be registered thanks to starpu_top_register_parameter_boolean(), starpu_top_register_parameter_integer(), starpu_top_register_parameter_float(), e.g.:

float alpha;
starpu_top_register_parameter_float("alpha", &alpha, 0, 10, modif_hook);

modif_hook is a function which will be called when the parameter is being modified, it can for instance print the new value:

void modif_hook(struct starpu_top_param *d)
{
fprintf(stderr,"%s has been modified: %f\n", d->name, alpha);
}

Task schedulers should notify StarPU-Top when it has decided when a task will be scheduled, so that it can show it in its Gantt chart, for instance:

Starting StarPU-Top (StarPU-Top is started via the binary starpu_top) and the application can be done in two ways:

• The application is started by hand on some machine (and thus already waiting for the start event). In the Preference dialog of StarPU-Top, the SSH checkbox should be unchecked, and the hostname and port (default is 2011) on which the application is already running should be specified. Clicking on the connection button will thus connect to the already-running application.
• StarPU-Top is started first, and clicking on the connection button will start the application itself (possibly on a remote machine). The SSH checkbox should be checked, and a command line provided, e.g.:

$ssh myserver STARPU_SCHED=dmda ./application  If port 2011 of the remote machine can not be accessed directly, an ssh port bridge should be added: $ ssh -L 2011:localhost:2011 myserver STARPU_SCHED=dmda ./application


and "localhost" should be used as IP Address to connect to.

# Task And Worker Profiling

A full example showing how to use the profiling API is available in the StarPU sources in the directory examples/profiling/.

/* We will destroy the task structure by hand so that we can
* query the profiling info before the task is destroyed. */
/* Submit and wait for completion (since synchronous was set to 1) */
/* The task is finished, get profiling information */
/* How much time did it take before the task started ? */
double delay += starpu_timing_timespec_delay_us(&info->submit_time, &info->start_time);
/* How long was the task execution ? */
double length += starpu_timing_timespec_delay_us(&info->start_time, &info->end_time);
/* We no longer need the task structure */
/* Display the occupancy of all workers during the test */
int worker;
for (worker = 0; worker < starpu_worker_get_count(); worker++)
{
struct starpu_profiling_worker_info worker_info;
int ret = starpu_profiling_worker_get_info(worker, &worker_info);
double total_time = starpu_timing_timespec_to_us(&worker_info.total_time);
double executing_time = starpu_timing_timespec_to_us(&worker_info.executing_time);
double sleeping_time = starpu_timing_timespec_to_us(&worker_info.sleeping_time);
double overhead_time = total_time - executing_time - sleeping_time;
float executing_ratio = 100.0*executing_time/total_time;
float sleeping_ratio = 100.0*sleeping_time/total_time;
float overhead_ratio = 100.0 - executing_ratio - sleeping_ratio;
char workername[128];
starpu_worker_get_name(worker, workername, 128);
fprintf(stderr, "Worker %s:\n", workername);
fprintf(stderr, "\ttotal time: %.2lf ms\n", total_time*1e-3);
fprintf(stderr, "\texec time: %.2lf ms (%.2f %%)\n", executing_time*1e-3, executing_ratio);
fprintf(stderr, "\tblocked time: %.2lf ms (%.2f %%)\n", sleeping_time*1e-3, sleeping_ratio);
• Measured at runtime (model type STARPU_HISTORY_BASED). This assumes that for a given set of data input/output sizes, the performance will always be about the same. This is very true for regular kernels on GPUs for instance (<0.1% error), and just a bit less true on CPUs (~=1% error). This also assumes that there are few different sets of data input/output sizes. StarPU will then keep record of the average time of previous executions on the various processing units, and use it as an estimation. History is done per task size, by using a hash of the input and ouput sizes as an index. It will also save it in $STARPU_HOME/.starpu/sampling/codelets for further executions, and can be observed by using the tool starpu_perfmodel_display, or drawn by using the tool starpu_perfmodel_plot (Performance Model Calibration). The models are indexed by machine name. To share the models between machines (e.g. for a homogeneous cluster), use export STARPU_HOSTNAME=some_global_name. Measurements are only done when using a task scheduler which makes use of it, such as dmda. Measurements can also be provided explicitly by the application, by using the function starpu_perfmodel_update_history(). The following is a small code example. If e.g. the code is recompiled with other compilation options, or several variants of the code are used, the symbol string should be changed to reflect that, in order to recalibrate a new model from zero. The symbol string can even be constructed dynamically at execution time, as long as this is done before submitting any task using it. static struct starpu_perfmodel mult_perf_model = { .symbol = "mult_perf_model" }; struct starpu_codelet cl = { .cpu_funcs = { cpu_mult }, .cpu_funcs_name = { "cpu_mult" }, .nbuffers = 3, .modes = { STARPU_R, STARPU_R, STARPU_W }, /* for the scheduling policy to be able to use performance models */ .model = &mult_perf_model }; • Measured at runtime and refined by regression (model types STARPU_REGRESSION_BASED and STARPU_NL_REGRESSION_BASED). This still assumes performance regularity, but works with various data input sizes, by applying regression over observed execution times. STARPU_REGRESSION_BASED uses an a*n^b regression form, STARPU_NL_REGRESSION_BASED uses an a*n^b+c (more precise than STARPU_REGRESSION_BASED, but costs a lot more to compute). For instance, tests/perfmodels/regression_based.c uses a regression-based performance model for the function memset(). Of course, the application has to issue tasks with varying size so that the regression can be computed. StarPU will not trust the regression unless there is at least 10% difference between the minimum and maximum observed input size. It can be useful to set the environment variable STARPU_CALIBRATE to 1 and run the application on varying input sizes with STARPU_SCHED set to dmda scheduler, so as to feed the performance model for a variety of inputs. The application can also provide the measurements explictly by using the function starpu_perfmodel_update_history(). The tools starpu_perfmodel_display and starpu_perfmodel_plot can be used to observe how much the performance model is calibrated (Performance Model Calibration); when their output look good, STARPU_CALIBRATE can be reset to 0 to let StarPU use the resulting performance model without recording new measures, and STARPU_SCHED can be set to dmda to benefit from the performance models. If the data input sizes vary a lot, it is really important to set STARPU_CALIBRATE to 0, otherwise StarPU will continue adding the measures, and result with a very big performance model, which will take time a lot of time to load and save. For non-linear regression, since computing it is quite expensive, it is only done at termination of the application. This means that the first execution of the application will use only history-based performance model to perform scheduling, without using regression. • Another type of model is STARPU_MULTIPLE_REGRESSION_BASED, which is based on multiple linear regression. In this model, the user defines both the relevant parameters and the equation for computing the task duration. are the parameters of the task, added at the task creation. These need to be extracted by the cl_perf_func function, which should be defined by the user. are the exponents defined by the user in model->combinations table. Finally, coefficients are computed automatically by the StarPU at the end of the execution, using least squares method of the dgels_ LAPACK function. examples/mlr/mlr.c example provides more details on the usage of STARPU_MULTIPLE_REGRESSION_BASED models. Coefficients computation is done at the end of the execution, and the results are stored in standard codelet perfmodel files. Additional files containing the duration of task together with the value of each parameter are stored in .starpu/sampling/codelets/tmp/ directory. These files are reused when STARPU_CALIBRATE environment variable is set to 1, to recompute coefficients based on the current, but also on the previous executions. Additionally, when multiple linear regression models are disabled (using "--disable-mlr" configuration option) or when the model->combinations are not defined, StarPU will still write output files into .starpu/sampling/codelets/tmp/ to allow performing an analysis. This analysis typically aims at finding the most appropriate equation for the codelet and tools/starpu_mlr_analysis script provides an example of how to perform such study. • Provided as an estimation from the application itself (model type STARPU_COMMON and field starpu_perfmodel::cost_function), see for instance examples/common/blas_model.h and examples/common/blas_model.c. • Provided explicitly by the application (model type STARPU_PER_ARCH): either field starpu_perfmodel::arch_cost_function, or the fields .per_arch[arch][nimpl].cost_function have to be filled with pointers to functions which return the expected duration of the task in micro-seconds, one per architecture, see for instance tests/datawizard/locality.c For STARPU_HISTORY_BASED, STARPU_REGRESSION_BASED, and STARPU_NL_REGRESSION_BASED, the dimensions of task data (both input and output) are used as an index by default. STARPU_HISTORY_BASED uses a CRC hash of the dimensions as an index to distinguish histories, and STARPU_REGRESSION_BASED and STARPU_NL_REGRESSION_BASED use the total size as an index for the regression. The starpu_perfmodel::size_base and starpu_perfmodel::footprint fields however permit the application to override that, when for instance some of the data do not matter for task cost (e.g. mere reference table), or when using sparse structures (in which case it is the number of non-zeros which matter), or when there is some hidden parameter such as the number of iterations, or when the application actually has a very good idea of the complexity of the algorithm, and just not the speed of the processor, etc. The example in the directory examples/pi uses this to include the number of iterations in the base size. starpu_perfmodel::size_base should be used when the variance of the actual performance is known (i.e. bigger returned value is longer execution time), and thus particularly useful for STARPU_REGRESSION_BASED or STARPU_NL_REGRESSION_BASED. starpu_perfmodel::footprint can be used when the variance of the actual performance is unknown (irregular performance behavior, etc.), and thus only useful for STARPU_HISTORY_BASED. starpu_task_data_footprint() can be used as a base and combined with other parameters through starpu_hash_crc32c_be() for instance. StarPU will automatically determine when the performance model is calibrated, or rather, it will assume the performance model is calibrated until the application submits a task for which the performance can not be predicted. For STARPU_HISTORY_BASED, StarPU will require 10 (STARPU_CALIBRATE_MINIMUM) measurements for a given size before estimating that an average can be taken as estimation for further executions with the same size. For STARPU_REGRESSION_BASED and STARPU_NL_REGRESSION_BASED, StarPU will require 10 (STARPU_CALIBRATE_MINIMUM) measurements, and that the minimum measured data size is smaller than 90% of the maximum measured data size (i.e. the measurement interval is large enough for a regression to have a meaning). Calibration can also be forced by setting the STARPU_CALIBRATE environment variable to 1, or even reset by setting it to 2. How to use schedulers which can benefit from such performance model is explained in Task Scheduling Policies. The same can be done for task energy consumption estimation, by setting the field starpu_codelet::energy_model the same way as the field starpu_codelet::model. Note: for now, the application has to give to the energy consumption performance model a name which is different from the execution time performance model. The application can request time estimations from the StarPU performance models by filling a task structure as usual without actually submitting it. The data handles can be created by calling any of the functions starpu_*_data_register with a NULL pointer and -1 node and the desired data sizes, and need to be unregistered as usual. The functions starpu_task_expected_length() and starpu_task_expected_energy() can then be called to get an estimation of the task cost on a given arch. starpu_task_footprint() can also be used to get the footprint used for indexing history-based performance models. starpu_task_destroy() needs to be called to destroy the dummy task afterwards. See tests/perfmodels/regression_based.c for an example. # Data trace and tasks length It is possible to get statistics about tasks length and data size by using : $ starpu_fxt_data_trace filename [codelet1 codelet2 ... codeletn]

Where filename is the FxT trace file and codeletX the names of the codelets you want to profile (if no names are specified, starpu_fxt_data_trace will profile them all). This will create a file, data_trace.gp which can be executed to get a .eps image of these results. On the image, each point represents a task, and each color corresponds to a codelet.