Profiler PPX#
The profiler PPX is an OCaml preprocessing tool allowing to use the Profiler
functions in any part of the code. The PPX allows to choose at compile time if
we want to have the profiler enabled or not.
This guide is aimed at explaining how to use the PPX, not the Profiler
. For
explanations about the profiler, please look at this page.
Both the PPX rewriter (our specific instance defined in the Ppx_profiler module ) and the PPX, the mechanism used to preprocess files in OCaml, are called PPX in this document.
Why a Profiler PPX?#
After having created a profiler, you can create a wrapper around it in any file with
module Profiler = (val Profiler.wrap my_profiler)
That can be used like this
Profiler.aggregate_f ~verbosity:Info ("advertise mempool", []) @@ fun () ->
advertise pv_shell advertisable_mempool;
let* _res =
Profiler.aggregate_s ~verbosity:Info ("set mempool", []) @@ fun () ->
set_mempool pv_shell our_mempool
in ...
The issue with this direct approach is that it creates wrapper around functions that may hinder their functioning. This is not wanted since the profiler is only used by devs hence the use of a PPX that is controlled by an environment variable.
TEZOS_PPX_PROFILER=<value> make
Will preprocess the code before compiling. The value
field is described
Enabled drivers.
This will allow to preprocess
() [@profiler.record {verbosity = Info} "merge store"] ;
into
Profiler.record ~verbosity:Info ("merge store", []) ;
() ;
It should be noted that a Profiler
module has to be available and has to
have the signature of the Profiler.GLOBAL_PROFILER module
that
can be obtained with module Profiler = (val Profiler.wrap my_profiler)
.
Of course you can create any module with this signature but in case you didn’t
name it Profiler
(let’s say you name it My_profiler
) you’ll have to
declare your PPX attributes with a profiler_module
field:
() [@profiler.record {verbosity = Info; profiler_module = My_profiler} "merge store"] ;
This will be preprocessed into
My_profiler.record ~verbosity:Info ("merge store", []) ;
() ;
How to use this PPX?#
There are three types of functions in the Profiler library.
1. Inline functions#
These functions are (for details about them, look at the The Profiler module document)
aggregate : verbosity:verbosity -> string * metadata -> unit
mark : verbosity:verbosity -> string list * metadata -> unit
record : verbosity:verbosity -> string * metadata -> unit
stamp : verbosity:verbosity -> string * metadata -> unit
stop : unit -> unit
The PPX allows to replace
Profiler.stop ();
Profiler.record ~verbosity:Info ("merge store", []);
...
with
()
[@profiler.stop]
[@profiler.record {verbosity = Info} "merge store"] ;
...
You can also decompose it to be sure of the evaluation order:
() [@profiler.stop] ;
() [@profiler.record {verbosity = Info} "merge store"] ;
...
2. Wrapping functions#
These functions are:
aggregate_f : verbosity:verbosity -> string * metadata -> (unit -> 'a) -> 'a
aggregate_s : verbosity:verbosity -> string * metadata -> (unit -> 'a Lwt.t) -> 'a Lwt.t
record_f : verbosity:verbosity -> string * metadata -> (unit -> 'a) -> 'a
record_s : verbosity:verbosity -> string * metadata -> (unit -> 'a Lwt.t) -> 'a Lwt.t
span_f : verbosity:verbosity -> string list * metadata -> (unit -> 'a) -> 'a
span_s : verbosity:verbosity -> string list * metadata -> (unit -> 'a Lwt.t) -> 'a Lwt.t
The PPX allows to replace
(Profiler.record_f ~verbosity:Info ("read_test_line", []) @@ fun () -> read_test_line ())
...
with
(read_test_line () [@profiler.record_f {verbosity = Info} "read_test_line"])
...
3. Custom values#
This PPX library provides a special construct, which basically acts as a
#ifndef TEZOS_PPX_PROFILER
/ #else
:
expr_if_ppx_not_used [@profiler.overwrite expr_if_ppx_used]
This construct will be preprocessed as expr_if_ppx_not_used
if you are
not using the PPX, or expr_if_ppx_used
if you are.
If you want to write a custom function that will use the intial value instead of simply removing it from the code, there are two functions that allow you to use a wrapper.
Both profiler.wrap_f
and profiler.wrap_s
(the Lwt.t
variant)
allow you to wrap a delayed version of the initial value (using
fun () -> ...
) with a custom function.
This will rewrite
let wrapper expr = do_something (); expr ()
let _ = expr [@profiler.wrap_f wrapper]
...
into
let wrapper expr = do_something (); expr ()
let _ = wrapper (fun () -> expr)
...
Structure of an attribute#
An attribute is a decoration attached to the syntax tree that allow the PPX to preprocess some part of the AST when reading them. It is composed of two parts:
[@attribute_id payload]
An attribute is attached to:
@
: the closest node (expression, patterns, etc.),let a = "preprocess this" [@attr_id payload]
, the attribute is attached to"preprocess this"
@@
: the closest block (type declaration, class fields, etc.),let preprocess this = "and this" [@@attr_id payload]
, the attribute is attached to the whole value binding@@@
: floating attributes are not used here
The grammar for attributes can be found in this page.
In the case of our PPX, the expected values are the following.
attribute_id
#
Allows to know the kind of functions we want to use (like @profiler.mark
or
@profiler.record_s
) and to link our PPX to all the attribute_ids
it can
handle. The use of profiler.
allows to make sure we don’t have any conflict
with another PPX.
payload
#
The payload is made of two parts, the first one being optional:
payload ::= record? args
record ::= { fields }
fields ::= field ; fields | empty
field ::=
| verbosity = (Notice | Info | Debug)
| profiler_module = module_ident
| metadata = <(string * string) list>
| driver_ids = <(Prometheus | OpenTelemetry | Text | Json) list>
args ::= <string> | <string list> | <function application> | ident | empty
As an example:
f x [@profiler.aggregate_s {verbosity = Info} g y z] ;
g x [@profiler.span_f {verbosity = Debug; profiler_module = Prof} "label"]
...
will be preprocessed as
Profiler.aggregate_s ~verbosity:Info (g y z) @@ f x ;
Prof.span_f ~verbosity:Debug ("label", []) @@ g x
...
Enabled drivers#
When enabling the ppx with TEZOS_PPX_PROFILER=<value>
, value
can have
two possible types:
A dummy one, all attributes will be preprocessed except the ones with a non-empty
driver_ids
fieldA list of driver ids like
prometheus; opentelemetry
that will allow to preprocess attributes:with an empty
driver_ids
fieldwith a
driver_ids
field where one of the driver ids is also present invalue
Adding functionalities#
To add a function that needs to be accepted by our PPX (let’s say we want to add
my_new_function
that was recently added to the Profiler
module) the
following files need to edited:
src/lib_ppx_profiler/rewriter.ml
:Add a
my_new_function_constant
toConstants
Add this constant to
Constants.constants
Add
My_new_function of content
toRewriter.t
Add a
my_new_function key location
constructor with its accepted payloads (usuallyKey.Apply
,Key.Ident
andKey.List
orKey.String
)
If this function needs to accept a new kind of payload (like an integer) you’ll need to edit
src/lib_ppx_profiler/key.ml
and theextract_key_from_payload
function inRewriter
(you can look at the ppxlib documentation)src/lib_ppx_profiler/expression.ml
where you’ll just need to addRewriter.my_new_function
to therewrite
function