Grafazos: Grafana dashboards for Octez node monitoring.

Grafazos: Grafana dashboards for Octez node monitoring.#

Grafazos allows to generate dashboards from jsonnet programs.

It relies on the metrics exposed by the Octez node, see Supported Open Metrics, to build grafana dashboards.

Resources#

Jsonnet#

To build the dashboards, you need to install jsonnet with version 0.18.0 minimum. We recommend to use google/go-jsonnet.

Grafonnet#

Grafonnet is the jsonnet library used to generate grafana dashboards.

Grafonnet should be installed using jsonnet-bundler.

To do so, run go install -a github.com/jsonnet-bundler/jsonnet-bundler/cmd/jb@latest and update your environment variables such as export GOPATH=$HOME/go and export PATH=$PATH:$GOPATH/bin.

Once jsonnet-bundler installed you can import dependencies (here grafonnet) by running the following:

jb install

Tools#

Build#

To create the dashboards:

make

The dashboards will be availbale in the grafazos/output/ folder.

You can also create a specific dashboard with

make dashboard_name

with dashboard_name one of:

  • compact: a single page compact dashboard which aims to give a brief overview of the node’s health,

  • basic: a simple dashboard displaying detailed node’s metrics,

  • logs: same as basic but also displaying node’s logs (thanks to Loki and promtail)

  • full: same as logs but also displaying hardware metrics (thanks to netdata)

  • dal-basic: a simple dashboard displaying some DAL node’s metrics,

To create the dashboards for a different branch

BRANCH=foo make

By default, the instance names label is set to instance. If you are using a particular label for the instance names, you can use NODE_INSTANCE_LABEL to set it:

NODE_INSTANCE_LABEL=my_instance_label

By default, the logs are filtered using the job label. If you need to use a different label for filtering logs (e.g., service), you can configure it with LOGS_LABEL:

LOGS_LABEL=service

By default, the RPC endpoints label is set to endpoint. If you need to use a different label for RPC endpoints (e.g., exported_endpoint), you can configure it with ENDPOINT_LABEL:

ENDPOINT_LABEL=exported_endpoint

If you need hardware metrics, note that, by default, the storage stat considered is the used space of the whole disk. Optionnally you can enable storage monitoring with filecheck:

STORAGE_MODE=filecheck

By default, all mountpoints are monitored. If you want to selectively change mountpoints to / and another specific mountpoint, you can enable and add it with the MOUNTPOINT variable: .. code-block:: shell

MOUNTPOINT=”/opt”

By default, the logs are fetched from Loki. If you want to use GCP logs, you can enable them with the LOGSRC variable, specifying the GCP project id with the GCP_PROJECT_ID variable and the data source unique id with GCP_DATASOURCE_UID.

LOGSRC="gcp"
GCP_PROJECT_ID="project_id"
GCP_DATASOURCE_UID="datasource_uid"

By default, dashboards are created without uid, resulting in a random one assigned by grafana. You can set a specific uid with the UID variable, which will be concatenated to a default uid set in each dasboard’s definition. If UID is set to an empty string, then the uid is set to the default uid value set for each dashboard.

UID="some_string"

By default, dashboards use a hardcoded Prometheus datasource. You can change the default datasource name with the DATASOURCE_DEFAULT variable:

DATASOURCE_DEFAULT=thanos

If you want to enable a dropdown menu to select from any available Prometheus datasource, set DATASOURCE_SELECTION to true:

DATASOURCE_SELECTION=true