Module Tezos_profiler.Profiler

Profiler

Summary

This profiling library declares a high-level interface meant to be used to instrument code in order to measure the time spent in the different parts in such a way to yield a (human-)processable report. This module declares a generic interface (driver) that will provide an API to the developer to instrument the code. When the profiling data is recorded, the abstracted profiler will feed it to its "plugged" backend (instance) which will process the different profiler's nodes in order to produce the reports. Reports may also be combined to interwine different components' traces.

The provided API is intentionally simplistic to simplify its usage. The basic usage is to call record <symbol> before the desired section to profile and stop () when we exit it. Nested calls are also supported and, given that the backend supports it, will be displayed as a callgraph. The API is also augmented with higher-level combinators in order to avoid mismatched stops and to support Lwt function calls.

type metadata = (string * string) list

Type used in order to (optionally) add information to a profiling call in the most generic way possible, allowing for different backends to need different data.

For instance, a prometheus-based profiler (which can't handle as many different metrics as the file-based ones) would rely on the "prometheus" key being present or not in metadata in order to determine if the function should be monitored or not.

type id = string * metadata

Name of a metric, with (possibly empty) metadata attached.

type ids = string list * metadata

Stack of names used for nested metrics (the head is the name of the metric, the second element is the name of the parent section, the third element the parent of the parent, etc...) with (possibly empty) metadata attached.

module IdMap : Stdlib.Map.S with type key = id

Types and utility functions

type time = {
  1. wall : float;
    (*

    Wall-clock time: total time elapsed.

    *)
  2. cpu : float;
    (*

    CPU time: time elapsed in the CPU.

    *)
}
type span =
  1. | Span of time
val zero_time : span
val (-*) : time -> time -> time
val (+*) : time -> time -> time
type verbosity =
  1. | Notice
  2. | Info
  3. | Debug

The level of detail of report sections. The driver can choose to use this information to skip or aggregate sections below a given level. The driver could also record everything, including the level of detail, and let a post processor skip or aggregate at display time.

type aggregated_node = {
  1. count : int;
  2. total : span;
  3. children : aggregated_node IdMap.t;
  4. node_verbosity : verbosity;
}

An aggregate node registers multiple calls to a section and sum their occurences and time. It also recursively aggregate its sub-aggregation nodes.

type seq_item = {
  1. start : time;
  2. duration : span;
  3. contents : report;
  4. item_verbosity : verbosity;
}

A sequence item registers one section with potential sub-reports and registers elapsed-time.

and report = {
  1. aggregated : aggregated_node IdMap.t;
  2. recorded : (id * seq_item) list;
}
val report_encoding : report Data_encoding.t
type (_, _) kind = ..
type view =
  1. | View : ('config, 'state) kind -> view
module type DRIVER = sig ... end
type 'a driver = (module DRIVER with type config = 'a)
type instance

Instance

A specific instance of a Driver implementation.

Example: this driver that writes text files in a unix filesystem will write them in this specific file with this specific level of details

val instance : 'a driver -> 'a -> instance

instance driver params will instantiate the driver with the given params

val time : instance -> time
val report : instance -> report option
val report_s : instance -> report Lwt.t
val close : instance -> unit
type profiler

Profiler

The profiler is an API that needs to be attached to a backend. A backend is composed of:

val plug : profiler -> instance -> unit

plug profiler instance plugs profiler to instance

val unplug : profiler -> instance -> unit

unplug profiler instance unplugs profiler from instance

val close_and_unplug : profiler -> instance -> unit

close_and_unplug profiler instance closes profiler and unplugs it from instance

val close_and_unplug_all : profiler -> unit

close_and_unplug_all profiler closes profiler and unplugs it from all instances it was plugged to

val plugged : profiler -> instance list

plugged profiler returns all the instances plugged to profiler

val record : profiler -> verbosity -> id -> unit

Open a sequence in the current sequence. If currently aggregating (not all aggregation scopes are closed), this has the same semantics as aggregate instead.

val aggregate : profiler -> verbosity -> id -> unit

Open an aggregation node in the current sequence.

val stop : profiler -> unit

Close the most recently opened sequence or aggregation scope.

val stamp : profiler -> verbosity -> id -> unit

Record a timestamp in the most recently opened sequence.

val mark : profiler -> verbosity -> ids -> unit

Count this event's occurences in the most recent sequence.

val span : profiler -> verbosity -> span -> ids -> unit

Sum the time spent in this event in the most recent sequence.

val inc : profiler -> report -> unit

Include a report in the current sequence.

val record_f : profiler -> verbosity -> id -> (unit -> 'a) -> 'a

record_f profiler verbosity label f will call:

record profiler verbosity name;
f ();
stop ();
val record_s : profiler -> verbosity -> id -> (unit -> 'a Lwt.t) -> 'a Lwt.t

Same as record_f but for Lwt function

val aggregate_f : profiler -> verbosity -> id -> (unit -> 'a) -> 'a

aggregate_f profiler verbosity label f will call:

aggregate profiler verbosity name;
f ();
stop ();
val aggregate_s : profiler -> verbosity -> id -> (unit -> 'a Lwt.t) -> 'a Lwt.t

Same as aggregate_f but for Lwt functions

val span_f : profiler -> verbosity -> ids -> (unit -> 'a) -> 'a

span_f profiler verbosity label_list f will compute span but specifically around f

val span_s : profiler -> verbosity -> ids -> (unit -> 'a Lwt.t) -> 'a Lwt.t

Same as span_f but for Lwt functions

val unplugged : unit -> profiler

unplugged () returns a new profiler

val main : profiler
module type GLOBAL_PROFILER = sig ... end
val wrap : profiler -> (module GLOBAL_PROFILER)

wrap profiler stores profiler in a GLOBAL_PROFILER module allowing to use the profiler functions without having to provide it as a parameter

type 'a section_maker = ('a * metadata) -> unit
val section_maker : ?verbosity:verbosity -> ('a -> 'a -> bool) -> ('a -> string) -> profiler -> 'a section_maker

section_maker equal to_string profiler Creates a function to open a new section (and close the one opened before) using record function when a new entity is encountered.

  • parameter verbosity
    • usual verbosity argument. Defaults to Notice
  • parameter equal
    • used for entities comparison.
  • parameter to_string
    • used for labeling the entity when using the record function.
  • parameter profiler
    • profiler instance used to track sections and record profiling data.