= Vert.x Micrometer Metrics

This project is an implementation of the Vert.x Metrics Service Provider Interface (SPI).

It uses link:http://micrometer.io/[Micrometer] for managing metrics and reporting to several backends.

== Features

* Vert.x core tools monitoring: TCP/HTTP client and servers, `link:../../apidocs/io/vertx/core/datagram/DatagramSocket.html[DatagramSocket]`,
`link:../../apidocs/io/vertx/core/eventbus/EventBus.html[EventBus]` and pools
* User defined metrics through Micrometer
* Reporting to https://www.influxdata.com/[InfluxDB], https://prometheus.io/[Prometheus] or JMX.

== InfluxDB

=== Prerequisites

Follow the https://docs.influxdata.com/influxdb/latest/introduction/getting_started/[instructions to get InfluxDb up and running].

=== Getting started

The modules _vertx-micrometer-metrics_ and _micrometer-registry-influx_ must be present in the classpath.

Maven users should add this to their project POM file:

[source,xml,subs="+attributes"]
----
<dependency>
  <groupId>io.vertx</groupId>
  <artifactId>vertx-micrometer-metrics</artifactId>
  <version>3.5.2.CR2</version>
</dependency>
<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-registry-influx</artifactId>
  <version>1.0.0</version>
</dependency>
----

And Gradle users, to their build file:

[source,groovy,subs="+attributes"]
----
compile 'io.vertx:vertx-micrometer-metrics:3.5.2.CR2'
compile 'io.micrometer:micrometer-registry-influx:1.0.0'
----

=== Configuration examples

Vert.x does not enable SPI implementations by default. You must enable metric collection in the Vert.x options.

[source,java]
----
Vertx vertx = Vertx.vertx(new VertxOptions().setMetricsOptions(
  new MicrometerMetricsOptions()
    .setInfluxDbOptions(new VertxInfluxDbOptions().setEnabled(true))
    .setEnabled(true)));
----

==== Using a specific URI and database name

[source,java]
----
Vertx vertx = Vertx.vertx(new VertxOptions().setMetricsOptions(
  new MicrometerMetricsOptions()
    .setInfluxDbOptions(new VertxInfluxDbOptions().setEnabled(true)
      .setUri("http://influxdb.example.com:8888")
      .setDb("sales-department"))
    .setEnabled(true)));
----

==== With authentication

[source,java]
----
Vertx vertx = Vertx.vertx(new VertxOptions().setMetricsOptions(
  new MicrometerMetricsOptions()
    .setInfluxDbOptions(new VertxInfluxDbOptions().setEnabled(true)
      .setUserName("username")
      .setPassword("password"))
    .setEnabled(true)));
----

== Prometheus

=== Prerequisites

Follow the https://prometheus.io/docs/prometheus/latest/getting_started/[instructions to get Prometheus up and running].

=== Getting started

The modules _vertx-micrometer-metrics_ and _micrometer-registry-prometheus_ must be present in the classpath.
You may also probably need _vertx-web_, to expose the metrics.

Maven users should add this to their project POM file:

[source,xml,subs="+attributes"]
----
<dependency>
  <groupId>io.vertx</groupId>
  <artifactId>vertx-micrometer-metrics</artifactId>
  <version>3.5.2.CR2</version>
</dependency>
<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-registry-prometheus</artifactId>
  <version>1.0.0</version>
</dependency>
<dependency>
  <groupId>io.vertx</groupId>
  <artifactId>vertx-web</artifactId>
  <version>3.5.2.CR2</version>
</dependency>
----

And Gradle users, to their build file:

[source,groovy,subs="+attributes"]
----
compile 'io.vertx:vertx-micrometer-metrics:3.5.2.CR2'
compile 'io.micrometer:micrometer-registry-prometheus:1.0.0'
compile 'io.vertx:vertx-web:3.5.2.CR2'
----

=== Configuration examples

Vert.x does not enable SPI implementations by default. You must enable metric collection in the Vert.x options

[source,java]
----
Vertx vertx = Vertx.vertx(new VertxOptions().setMetricsOptions(
  new MicrometerMetricsOptions()
    .setPrometheusOptions(new VertxPrometheusOptions().setEnabled(true))
    .setEnabled(true)));
----

==== Using an embedded HTTP server wih custom endpoint

[source,java]
----
Vertx vertx = Vertx.vertx(new VertxOptions().setMetricsOptions(
  new MicrometerMetricsOptions()
    .setPrometheusOptions(new VertxPrometheusOptions().setEnabled(true)
      .setStartEmbeddedServer(true)
      .setEmbeddedServerOptions(new HttpServerOptions().setPort(8080))
      .setEmbeddedServerEndpoint("/metrics/vertx"))
    .setEnabled(true)));
----

If the embedded server endpoint is not specified, it defaults to _/metrics_.

==== Binding metrics to an existing Vert.x router

[source,java]
----
Vertx vertx = Vertx.vertx(new VertxOptions().setMetricsOptions(
  new MicrometerMetricsOptions()
    .setPrometheusOptions(new VertxPrometheusOptions().setEnabled(true))
    .setEnabled(true)));

// Later on, creating a router
Router router = Router.router(vertx);
router.route("/metrics").handler(routingContext -> {
  PrometheusMeterRegistry prometheusRegistry = (PrometheusMeterRegistry) BackendRegistries.getDefaultNow();
  if (prometheusRegistry != null) {
    String response = prometheusRegistry.scrape();
    routingContext.response().end(response);
  } else {
    routingContext.fail(500);
  }
});
vertx.createHttpServer().requestHandler(router::accept).listen(8080);
----

== JMX

=== Getting started

The modules _vertx-micrometer-metrics_ and _micrometer-registry-jmx_ must be present in the classpath.

Maven users should add this to their project POM file:

[source,xml,subs="+attributes"]
----
<dependency>
  <groupId>io.vertx</groupId>
  <artifactId>vertx-micrometer-metrics</artifactId>
  <version>3.5.2.CR2</version>
</dependency>
<dependency>
  <groupId>io.micrometer</groupId>
  <artifactId>micrometer-registry-jmx</artifactId>
  <version>1.0.0</version>
</dependency>
----

And Gradle users, to their build file:

[source,groovy,subs="+attributes"]
----
compile 'io.vertx:vertx-micrometer-metrics:3.5.2.CR2'
compile 'io.micrometer:micrometer-registry-jmx:1.0.0'
----

=== Configuration examples

Vert.x does not enable SPI implementations by default. You must enable metric collection in the Vert.x options

[source,java]
----
Vertx vertx = Vertx.vertx(new VertxOptions().setMetricsOptions(
  new MicrometerMetricsOptions()
    .setJmxMetricsOptions(new VertxJmxMetricsOptions().setEnabled(true))
    .setEnabled(true)));
----

==== With step and domain

In Micrometer, `step` refers to the reporting period, in seconds. `domain` is the JMX domain under which
MBeans are registered.

[source,java]
----
Vertx vertx = Vertx.vertx(new VertxOptions().setMetricsOptions(
  new MicrometerMetricsOptions()
    .setJmxMetricsOptions(new VertxJmxMetricsOptions().setEnabled(true)
      .setStep(5)
      .setDomain("my.metrics.domain"))
    .setEnabled(true)));
----

== Advanced usage

Please refer to `io.vertx.micrometer.MicrometerMetricsOptions` for an exhaustive list of options.

=== Disable some metric domains

Restricting the Vert.x modules being monitored can be done using
`io.vertx.micrometer.MicrometerMetricsOptions#disabledMetricsCategories`.

For a full list of domains, see `io.vertx.micrometer.MetricsDomain`

=== User-defined metrics

The Micrometer registries are accessible, in order to create new metrics or fetch the existing ones.
By default, an unique registry is used and will be shared across the Vert.x instances of the JVM:

[source,java]
----
MeterRegistry registry = BackendRegistries.getDefaultNow();
----

It is also possible to have separate registries per Vertx instance, by giving a registry name in metrics options.
Then it can be retrieved specifically:

[source,java]
----
Vertx vertx = Vertx.vertx(new VertxOptions().setMetricsOptions(
  new MicrometerMetricsOptions()
    .setInfluxDbOptions(new VertxInfluxDbOptions().setEnabled(true)) // or VertxPrometheusOptions
    .setRegistryName("my registry")
    .setEnabled(true)));

// Later on:
MeterRegistry registry = BackendRegistries.getNow("my registry");
----

As an example, here is a custom timer that will track the execution time of a piece of code that is regularly called:

[source,java]
----
MeterRegistry registry = BackendRegistries.getDefaultNow();
Timer timer = Timer
  .builder("my.timer")
  .description("a description of what this timer does")
  .register(registry);

vertx.setPeriodic(1000, l -> {
  timer.record(() -> {
    // Running here some operation to monitor
  });
});
----

For more examples, documentation about the Micrometer registry and how to create metrics, check
link:http://micrometer.io/docs/concepts#_registry[Micrometer doc].

=== Other instrumentation

Since plain access to Micrometer registries is provided, it is possible to leverage the Micrometer API.
For instance, to instrument the JVM:

[source,java]
----
MeterRegistry registry = BackendRegistries.getDefaultNow();

new ClassLoaderMetrics().bindTo(registry);
new JvmMemoryMetrics().bindTo(registry);
new JvmGcMetrics().bindTo(registry);
new ProcessorMetrics().bindTo(registry);
new JvmThreadMetrics().bindTo(registry);
----

_From link:http://micrometer.io/docs/ref/jvm[Micrometer documentation]._

=== Label matchers

The labels (aka tags, or fields...) can be configured through the use of matchers. Here is an example
to whitelist HTTP server metrics per host name and port:

[source,java]
----
Vertx vertx = Vertx.vertx(new VertxOptions().setMetricsOptions(
  new MicrometerMetricsOptions()
    .setPrometheusOptions(new VertxPrometheusOptions().setEnabled(true))
    .addLabelMatch(new Match()
      // Restrict HTTP server metrics to those with label "local=localhost:8080" only
      .setDomain(MetricsDomain.HTTP_SERVER)
      .setLabel("local")
      .setValue("localhost:8080"))
    .setEnabled(true)));
----

Matching rules can work on exact strings or regular expressions (the former is more performant).
When a pattern matches, the value can also be renamed with an alias. By playing with regex and aliases it is possible
to ignore a label partitioning:

[source,java]
----
Vertx vertx = Vertx.vertx(new VertxOptions().setMetricsOptions(
  new MicrometerMetricsOptions()
    .setPrometheusOptions(new VertxPrometheusOptions().setEnabled(true))
    .addLabelMatch(new Match()
      // Set all values for "remote" label to "_", for all domains. In other words, it's like disabling the "remote" label.
      .setLabel("remote")
      .setType(MatchType.REGEX)
      .setValue(".*")
      .setAlias("_"))
    .setEnabled(true)));
----

Here, any value for the label "remote" will be replaced with "_".

Sometimes, having some labels results in a high cardinality of label values, which can cause troubles / bad
performances on the metrics backend.
This is often the case with the _remote_ label on server metrics.
For that reason, there are rules in the default metrics options to ignore it (one for HTTP server, one for Net server metrics).
It is still possible to stop ignoring it by clearing the label matchers:

[source,java]
----
Vertx vertx = Vertx.vertx(new VertxOptions().setMetricsOptions(
  new MicrometerMetricsOptions()
    .setPrometheusOptions(new VertxPrometheusOptions().setEnabled(true))
    .setLabelMatchs(new ArrayList<>())
    .setEnabled(true)));
----

Label matching uses Micrometer's `MeterFilter` under the hood. This API can be accessed directly as well:

[source,java]
----
MeterRegistry registry = BackendRegistries.getDefaultNow();

registry.config().meterFilter(MeterFilter.ignoreTags("address", "remote"))
  .meterFilter(MeterFilter.renameTag("vertx.verticle", "deployed", "instances"));
----

_See also link:http://micrometer.io/docs/concepts#_meter_filters[other examples]._

=== Snapshots

A `io.vertx.micrometer.MetricsService` can be created out of a `link:../../apidocs/io/vertx/core/metrics/Measured.html[Measured]` object
in order to take a snapshot of its related metrics and measurements.
The snapshot is returned as a `link:../../apidocs/io/vertx/core/json/JsonObject.html[JsonObject]`.

A well known _Measured_ object is simply `link:../../apidocs/io/vertx/core/Vertx.html[Vertx]`:

[source,java]
----
MetricsService metricsService = MetricsService.create(vertx);
JsonObject metrics = metricsService.getMetricsSnapshot();
System.out.println(metrics);
----

Other components, such as an `link:../../apidocs/io/vertx/core/eventbus/EventBus.html[EventBus]` or a `link:../../apidocs/io/vertx/core/http/HttpServer.html[HttpServer]` are
measurable:

[source,java]
----
HttpServer server = vertx.createHttpServer();
MetricsService metricsService = MetricsService.create(server);
JsonObject metrics = metricsService.getMetricsSnapshot();
System.out.println(metrics);
----

Finally it is possible to filter the returned metrics from their base names:

[source,java]
----
MetricsService metricsService = MetricsService.create(vertx);
// Client + server
JsonObject metrics = metricsService.getMetricsSnapshot("vertx.http");
System.out.println(metrics);
----

== Vert.x core tools metrics

This section lists all the metrics generated by monitoring the Vert.x core tools.

=== Net Client

[cols="15,50,35", options="header"]
|===
|Metric type
|Metric name
|Description

|Gauge
|`vertx_net_client_connections{local=<local address>,remote=<remote address>}`
|Number of connections to the remote host currently opened.

|Summary
|`vertx_net_client_bytesReceived{local=<local address>,remote=<remote address>}`
|Number of bytes received from the remote host.

|Summary
|`vertx_net_client_bytesSent{local=<local address>,remote=<remote address>}`
|Number of bytes sent to the remote host.

|Counter
|`vertx_net_client_errors{local=<local address>,remote=<remote address>,class=<class>}`
|Number of errors.

|===

=== HTTP Client

[cols="15,50,35", options="header"]
|===
|Metric type
|Metric name
|Description

|Gauge
|`vertx_http_client_connections{local=<local address>,remote=<remote address>}`
|Number of connections to the remote host currently opened.

|Summary
|`vertx_http_client_bytesReceived{local=<local address>,remote=<remote address>}`
|Number of bytes received from the remote host.

|Summary
|`vertx_http_client_bytesSent{local=<local address>,remote=<remote address>}`
|Number of bytes sent to the remote host.

|Counter
|`vertx_http_client_errors{local=<local address>,remote=<remote address>,class=<class>}`
|Number of errors.

|Gauge
|`vertx_http_client_requests{local=<local address>,remote=<remote address>}`
|Number of requests waiting for a response.

|Counter
|`vertx_http_client_requestCount{local=<local address>,remote=<remote address>,method=<http method>}`
|Number of requests sent.

|Timer
|`vertx_http_client_responseTime{local=<local address>,remote=<remote address>}`
|Response time.

|Counter
|`vertx_http_client_responseCount{local=<local address>,remote=<remote address>,code=<response code>}`
|Number of received responses.

|Gauge
|`vertx_http_client_wsConnections{local=<local address>,remote=<remote address>}`
|Number of websockets currently opened.

|===

=== Datagram socket

[cols="15,50,35", options="header"]
|===
|Metric type
|Metric name
|Description

|Summary
|`vertx_datagram_bytesReceived{local=<local>,remote=<remote>}`
|Total number of bytes received on the `<host>:<port>` listening address.

|Summary
|`vertx_datagram_bytesSent{remote=<remote>}`
|Total number of bytes sent to the remote host.

|Counter
|`vertx_datagram_errors{class=<class>}`
|Total number of errors.

|===

=== Net Server

[cols="15,50,35", options="header"]
|===
|Metric type
|Metric name
|Description

|Gauge
|`vertx_net_server_connections{local=<local address>}`
|Number of opened connections to the Net Server.

|Summary
|`vertx_net_server_bytesReceived{local=<local address>}`
|Number of bytes received by the Net Server.

|Summary
|`vertx_net_server_bytesSent{local=<local address>}`
|Number of bytes sent by the Net Server.

|Counter
|`vertx_net_server_errors{local=<local address>,class=<class>}`
|Number of errors.

|===

=== HTTP Server

[cols="15,50,35", options="header"]
|===
|Metric type
|Metric name
|Description

|Gauge
|`vertx_http_server_connections{local=<local address>}`
|Number of opened connections to the HTTP Server.

|Summary
|`vertx_http_server_bytesReceived{local=<local address>}`
|Number of bytes received by the HTTP Server.

|Summary
|`vertx_http_server_bytesSent{local=<local address>}`
|Number of bytes sent by the HTTP Server.

|Counter
|`vertx_http_server_errors{local=<local address>,class=<class>}`
|Number of errors.

|Gauge
|`vertx_http_server_requests{local=<local address>}`
|Number of requests being processed.

|Counter
|`vertx_http_server_requestCount{local=<local address>,method=<http method>,code=<response code>}`
|Number of processed requests.

|Counter
|`vertx_http_server_requestResetCount{local=<local address>}`
|Number of requests reset.

|Timer
|`vertx_http_server_processingTime{local=<local address>}`
|Request processing time.

|Gauge
|`vertx_http_client_wsConnections{local=<local address>}`
|Number of websockets currently opened.

|===

=== Event Bus

[cols="15,50,35", options="header"]
|===
|Metric type
|Metric name
|Description

|Gauge
|`vertx_eventbus_handlers{address=<address>}`
|Number of event bus handlers in use.

|Counter
|`vertx_eventbus_errors{address=<address>,class=<class>}`
|Number of errors.

|Summary
|`vertx_eventbus_bytesWritten{address=<address>}`
|Total number of bytes sent while sending messages to event bus cluster peers.

|Summary
|`vertx_eventbus_bytesRead{address=<address>}`
|Total number of bytes received while reading messages from event bus cluster peers.

|Gauge
|`vertx_eventbus_pending{address=<address>,side=<local/remote>}`
|Number of messages not processed yet. One message published will count for `N` pending if `N` handlers
are registered to the corresponding address.

|Counter
|`vertx_eventbus_published{address=<address>,side=<local/remote>}`
|Number of messages published (publish / subscribe).

|Counter
|`vertx_eventbus_sent{address=<address>,side=<local/remote>}`
|Number of messages sent (point-to-point).

|Counter
|`vertx_eventbus_received{address=<address>,side=<local/remote>}`
|Number of messages received.

|Counter
|`vertx_eventbus_delivered{address=<address>,side=<local/remote>}`
|Number of messages delivered to handlers.

|Counter
|`vertx_eventbus_replyFailures{address=<address>,failure=<failure name>}`
|Number of message reply failures.

|Timer
|`vertx_eventbus_processingTime{address=<address>}`
|Processing time for handlers listening to the `address`.

|===

== Vert.x pool metrics

This section lists all the metrics generated by monitoring Vert.x pools.

There are two types currently supported:

* _worker_ (see `link:../../apidocs/io/vertx/core/WorkerExecutor.html[WorkerExecutor]`)
* _datasource_ (created with Vert.x JDBC client)

NOTE: Vert.x creates two worker pools upfront, _worker-thread_ and _internal-blocking_.

[cols="15,50,35", options="header"]
|===
|Metric type
|Metric name
|Description

|Timer
|`vertx_pool_queue_delay{pool_type=<type>,pool_name=<name>}`
|Time waiting for a resource (queue time).

|Gauge
|`vertx_pool_queue_size{pool_type=<type>,pool_name=<name>}`
|Number of elements waiting for a resource.

|Timer
|`vertx_pool_usage{pool_type=<type>,pool_name=<name>}`
|Time using a resource (i.e. processing time for worker pools).

|Gauge
|`vertx_pool_inUse{pool_type=<type>,pool_name=<name>}`
|Number of resources used.

|Counter
|`vertx_pool_completed{pool_type=<type>,pool_name=<name>}`
|Number of elements done with the resource (i.e. total number of tasks executed for worker pools).

|Gauge
|`vertx_pool_ratio{pool_type=<type>,pool_name=<name>}`
|Pool usage ratio, only present if maximum pool size could be determined.

|===

== Verticle metrics

[cols="15,50,35", options="header"]
|===
|Metric type
|Metric name
|Description

|Gauge
|`vertx_verticle_deployed{name=<name>}`
|Number of verticle instances deployed.

|===