Profiler (octez-libs.Tezos_profiler.Profiler)

Profiler

Summary

This profiling library declares a high-level interface meant to be used to instrument code in order to measure the time spent in the different parts in such a way to yield a (human-)processable report. This module declares a generic interface (driver) that will provide an API to the developer to instrument the code. When the profiling data is recorded, the abstracted profiler will feed it to its "plugged" backend (instance) which will process the different profiler's nodes in order to produce the reports. Reports may also be combined to interwine different components' traces.

The provided API is intentionally simplistic to simplify its usage. The basic usage is to call record <symbol> before the desired section to profile and stop () when we exit it. Nested calls are also supported and, given that the backend supports it, will be displayed as a callgraph. The API is also augmented with higher-level combinators in order to avoid mismatched stops and to support Lwt function calls.

type metadata = (string * string) list

Type used in order to (optionally) add information to a profiling call in the most generic way possible, allowing for different backends to need different data.

For instance, a prometheus-based profiler (which can't handle as many different metrics as the file-based ones) would rely on the "prometheus" key being present or not in metadata in order to determine if the function should be monitored or not.

type id = string * metadata

Name of a metric, with (possibly empty) metadata attached.

type ids = string list * metadata

Stack of names used for nested metrics (the head is the name of the metric, the second element is the name of the parent section, the third element the parent of the parent, etc...) with (possibly empty) metadata attached.

module IdMap : Stdlib.Map.S with type key = id

Types and utility functions

type time = {

wall : float;
(*
Wall-clock time: total time elapsed.
*)
cpu : float option;
(*
CPU time: time elapsed in the CPU.
*)

}

type span =

| Span of time

val compute_cpu : cpu:bool option -> bool

compute_cpu ~cpu returns true if CPU profiling should be enabled.

Computing CPU times is slow, if the overhead when profiling is too important, consider disabling cpu profiling globally or locally on functions that are called a lot.

This depends on the value of the CPU_PROFILING environment variable and cpu.

CPU_PROFILING value dictates what kind of value is returned:

true means cpu profiling will be enabled everywhere
false means cpu profiling will be disabled everywhere
default_true means cpu profiling will be enabled except for the profiling parts where cpu:false is explicit
default_false means cpu profiling will be disabled except for the profiling parts where cpu:true is explicit.

val zero_time : cpu:bool option -> span

val (-*) : time -> time -> time

val (+*) : time -> time -> time

type verbosity =

| Notice
| Info
| Debug

The level of detail of report sections. The driver can choose to use this information to skip or aggregate sections below a given level. The driver could also record everything, including the level of detail, and let a post processor skip or aggregate at display time.

type aggregated_node = {

count : int;
total : span;
children : aggregated_node IdMap.t;
node_verbosity : verbosity;

}

An aggregate node registers multiple calls to a section and sum their occurences and time. It also recursively aggregate its sub-aggregation nodes.

type seq_item = {

start : time;
duration : span;
contents : report;
item_verbosity : verbosity;

}

A sequence item registers one section with potential sub-reports and registers elapsed-time.

and report = {

aggregated : aggregated_node IdMap.t;
recorded : (id * seq_item) list;

}

val report_encoding : report Data_encoding.t

type (_, _) kind = ..

type view =

| View : ('config, 'state) kind -> view

module type DRIVER = sig ... end

type 'a driver = (module DRIVER with type config = 'a)

type instance

Instance

A specific instance of a Driver implementation.

Example: this driver that writes text files in a unix filesystem will write them in this specific file with this specific level of details

val instance : 'a driver -> 'a -> instance

instance driver params will instantiate the driver with the given params

val time : cpu:bool option -> instance -> time

val report : cpu:bool option -> instance -> report option

val report_s : instance -> report Lwt.t

val close : instance -> unit

type profiler

Profiler

The profiler is an API that needs to be attached to a backend. A backend is composed of:

a Driver
a Instance

val plug : profiler -> instance -> unit

plug profiler instance plugs profiler to instance

val unplug : profiler -> instance -> unit

unplug profiler instance unplugs profiler from instance

val close_and_unplug : profiler -> instance -> unit

close_and_unplug profiler instance closes profiler and unplugs it from instance

val close_and_unplug_all : profiler -> unit

close_and_unplug_all profiler closes profiler and unplugs it from all instances it was plugged to

val plugged : profiler -> instance list

plugged profiler returns all the instances plugged to profiler

val record : cpu:bool option -> profiler -> verbosity -> id -> unit

Open a sequence in the current sequence. If currently aggregating (not all aggregation scopes are closed), this has the same semantics as aggregate instead.

val aggregate : cpu:bool option -> profiler -> verbosity -> id -> unit

Open an aggregation node in the current sequence.

val stop : profiler -> unit

Close the most recently opened sequence or aggregation scope.

val stamp : cpu:bool option -> profiler -> verbosity -> id -> unit

Record a timestamp in the most recently opened sequence.

val mark : profiler -> verbosity -> ids -> unit

Count this event's occurences in the most recent sequence.

val span : cpu:bool option -> profiler -> verbosity -> span -> ids -> unit

Sum the time spent in this event in the most recent sequence.

val inc : profiler -> report -> unit

Include a report in the current sequence.

val record_f : 
  cpu:bool option ->
  profiler ->
  verbosity ->
  id ->
  (unit -> 'a) ->
  'a

record_f profiler verbosity label f will call:

  record profiler verbosity name;
  f ();
  stop ();

val record_s : 
  cpu:bool option ->
  profiler ->
  verbosity ->
  id ->
  (unit -> 'a Lwt.t) ->
  'a Lwt.t

Same as record_f but for Lwt function

val aggregate_f : 
  cpu:bool option ->
  profiler ->
  verbosity ->
  id ->
  (unit -> 'a) ->
  'a

aggregate_f profiler verbosity label f will call:

  aggregate profiler verbosity name;
  f ();
  stop ();

val aggregate_s : 
  cpu:bool option ->
  profiler ->
  verbosity ->
  id ->
  (unit -> 'a Lwt.t) ->
  'a Lwt.t

Same as aggregate_f but for Lwt functions

val span_f : 
  cpu:bool option ->
  profiler ->
  verbosity ->
  ids ->
  (unit -> 'a) ->
  'a

span_f profiler verbosity label_list f will compute span but specifically around f

val span_s : 
  cpu:bool option ->
  profiler ->
  verbosity ->
  ids ->
  (unit -> 'a Lwt.t) ->
  'a Lwt.t

Same as span_f but for Lwt functions

val unplugged : unit -> profiler

unplugged () returns a new profiler

val main : profiler

module type GLOBAL_PROFILER = sig ... end

val wrap : profiler -> (module GLOBAL_PROFILER)

wrap profiler stores profiler in a GLOBAL_PROFILER module allowing to use the profiler functions without having to provide it as a parameter

type 'a section_maker = ('a * metadata) -> unit

val section_maker : 
  ?verbosity:verbosity ->
  ?cpu:bool ->
  ('a -> 'a -> bool) ->
  ('a -> string) ->
  profiler ->
  'a section_maker

section_maker equal to_string profiler Creates a function to open a new section (and close the one opened before) using record function when a new entity is encountered.

parameter verbosity
- usual verbosity argument. Defaults to Notice

parameter equal
- used for entities comparison.

parameter to_string
- used for labeling the entity when using the record function.

parameter profiler
- profiler instance used to track sections and record profiling data.