[jmx-scraper] Create weaver model for jmx-scraper metrics #2072

[mcmho]

Description:
Feature / Improvement: use weaver model to describe metrics so that they can be standardized.

• created metrics.yaml weaver model for jvm metrics
• created attriutes.yaml
• tested with weaver registry generate

Testing:

• tested with weaver registry generate; metadata.yaml is successfully generated and content is valided

Documentation:

• customer facing docs not yet needed

Outstanding items:

• once this looks good, there will be other models to work on for other target systems.

[atoulme]

LGTM - please look at the CI?

[SylvainJuge]

I am not familiar with weaver, does it means that the jmx scraper yaml configuration would be now generated from registry+template ? Maybe we could modify the yaml format to make this easier or even avoid having to use a template.

Do you have an high level overview of what this change helps to achieve?

Also, the metrics descriptions are part of what is being tested in unit tests, so when changing them you'll have to update the test code as well.

[mcmho]

Thanks all. sorry for the delay! The tests are fixed now.

Re: @SylvainJuge
yes, the rule configuration will be generated from jinja2 template and registry.

Perhaps @atoulme can chime in since he created the original request. As far as I understand, one of the main benefits coming out of this is standardization of metrics. Ultimately, we can have a centralized registry of metrics. That would become the single source of truth / semconv. I think that's what @atoulme has in mind "The idea is that eventually we can move some of the weaver models to semconv, as metric references."

[SylvainJuge]

If I understand this correctly, this PR does the following:

• makes the weaver yaml descriptor the source of truth to define JMX metrics (limited to the JVM target system for now)
• adds a jinja template to translate from the weaver yaml to the JMX metrics yaml

With that, modifying the JMX metrics yaml would require expertise in jinja and in running weaver with a makefile, which is not how things are usually done in this java-only project.

What I think would be simpler here is to keep the JMX metrics yaml as source of truth and then generate the weaver yaml files from it as part of the build process.

The benefits I see for this approach would be the following:

• it's easier to add extra attributes in JMX yaml for the weaver yaml generation than using jinja template to do the opposite
• keep the implementation 100% in Java
• easy to integrate into current gradle build
• could be included directly into the jmx-scraper CLI so that end-users could also do the same with their custom JMX metrics definitions.
• adding support for other target systems would only require to update their jmx yaml (with the few new extra fields), without having to deal with a dedicated template for each and thus handle them all at once.
• we could still use the weaver yaml and templates to generate and update the documentation markdown, for example as part of the CI.

I will need to prototype this to validate this is a viable approach, I will try to do this in the following weeks. This will need to happen in the instrumentation repository as the jmx-metrics implementation is part of it.

In addition to that, we also need to deal with the following extra challenges:

• jmx-scraper can use two sets of metrics: the "legacy" ones in jmx-scraper and the ones from "instrumentation" that are inherited from instrumentation (depends on the value of otel.jmx.target.source configuration), do we have a way to describe things that depend on configuration in the weaver description ?
• the jmx metrics definitions that are in instrumentation should have their definitions in the java instrumentation repository, but we can still generate the effective weaver yaml (including what is inherited).

Regarding the ability to push JMX metrics into semconv, this would be awesome but also challenging as product implementation might change across versions and would also require to create semconv SIG(s). I'd be happy to participate to that initiative when it starts.

[SylvainJuge]

I've opened open-telemetry/opentelemetry-java-instrumentation#15196 to attempt to implement this directly into java instrumentation.

[atoulme]

With that, modifying the JMX metrics yaml would require expertise in jinja and in running weaver with a makefile, which is not how things are usually done in this java-only project.

Please upvote and consider participating in open-telemetry/weaver#792

I'm sure we can work out a gradle task to run a docker container, might take a bit to get the kinks out.

The choice of jinja2 as a templating language seems like the least of all choices at this point, but please feel free to offer alternatives in the weaver repository?

What I think would be simpler here is to keep the JMX metrics yaml as source of truth and then generate the weaver yaml files from it as part of the build process.

It can work, but the weaver model is richer than the JMX metrics yaml though.

[jsuereth]

You could put an annotation on the metric and use it here if you wanted.

So your metrics.yaml would be something like:

groups:
  - id: jvm.memory.head.committed
    type: metric
    ...
    annotations:
    - jmx_name: HeapMemoryUsage.committed

Then here you could have the mapping generically pull the annotation:

{{ m.annotatiions.jmx_name }}:
  metric: {{ m.id | ... }
  ...

[jsuereth]

You could even go further and record the Bean / property and group-by bean, etc.

[SylvainJuge]

I think the weaver yaml should be used as a "contract" to describe and document what signals are captured by a specific compolent, this allows to automate documentation and maybe generate other artifacts derived from it.

In a sense, this is quite similar to the work around metadata in instrumentation, what is needed is the ability to provide an up-to-date weaver model to allow consumers to diff/migrate using weaver tool.

My intent here would be to provide the following:

• create a reusable tool included in instrumentation/jmx-metrics that allows to generate weaver yaml from any JMX metrics yaml definition jmx metrics generate weaver yaml from jmx metrics yaml opentelemetry-java-instrumentation#15196
• use this tool to generate metrics definitions for current and past versions of JMX metrics in this repository, those would be checked-in git repo (to be later consumable by weaver)
• integrate this tool within build to automate the process to update generated weaver yaml
• do the same in contrib/jmx-scraper for the "legacy" metrics definitions, this will also include the metrics definitions inherited from instrumentation.

In addition, the users of JMX metrics are also likely to provide their own metrics definitions, so having the ability to generate the weaver yaml for them would also provide the ability for end-users to also generate documentation, versioning and migration of their custom data.

I don't think using weaver to generate the JMX metrics yaml provides anything beyond generating one yaml from another, it just makes the maintenance of those JMX metrics yaml extra complicated due to the templating, also it does not allow to provide to end-users the ability to generate their own weaver yaml. In a sense, it would be trying to re-implement the JMX metrics yaml into the weaver yaml (we could even push it further by making JMX metrics reuse weaver yaml format, but that would be a breaking change and likely an extra challenge to document and maintain and help users migrating.

However, I'm not against using the weaver itself, for example I agree that it would make sense to use it to generate documentation from the weaver yaml, this is currently done manually.

I plan to experiment a bit this week trying to implement open-telemetry/opentelemetry-java-instrumentation#15196 as I think this is a simpler approach.

[jackshirazi]

Bearing in mind that

• JMX is Java specific
• the metrics generated for the most part from this are for Java specific tech (very few metrics are generic non-Java system level)
• that the the configurers will be looking at predominantly Java specific documentation to decide what metrics to setup
• there is already an existing user base

I think moving the primary configuration to new format is the wrong approach. Having something that converts between two formats is great, but it shouldn't require an extra tool to use the jmx-scraper