NIFI-14077 Add ProcessGroup Performance Metrics to Prometheus

[esecules]

Summary

NIFI-14077

• Add ProcessGroup Performance Metrics to Prometheus if performance metric collection is available
• Add total task duration metric to each process group, not just the root process group
• Add allowable values to the API docs for the producer and includedRegistries parameters so users know what there is to query without going to the source code.

Tracking

Please complete the following tracking steps prior to pull request creation.

Issue Tracking

• Apache NiFi Jira issue created

Pull Request Tracking

• Pull Request title starts with Apache NiFi Jira issue number, such as NIFI-00000
• Pull Request commit message starts with Apache NiFi Jira issue number, as such NIFI-00000

Pull Request Formatting

• Pull Request based on current revision of the main branch
• Pull Request refers to a feature branch with one commit containing changes

Verification

Please indicate the verification steps performed prior to pull request creation.

Build

• Build completed using mvn clean install -P contrib-check
  • JDK 21

Licensing

(no new deps)

• New dependencies are compatible with the Apache License 2.0 according to the License Policy
• New dependencies are documented in applicable LICENSE and NOTICE files

Documentation

• Documentation formatting appears as expected in rendered files

[esecules]

I suspect the macos system test is flakey so I am rerunning it on my fork https://github.com/esecules/nifi/actions/runs/12285436758?pr=3

EDIT:
The run passed on my fork hopefully it does the same below (thanks for the speedy retry @exceptionfactory!)

[exceptionfactory]

I suspect the macos system test is flakey so I am rerunning it on my fork https://github.com/esecules/nifi/actions/runs/12285436758?pr=3

EDIT: The run passed on my fork hopefully it does the same below (thanks for the speedy retry @exceptionfactory!)

You're welcome, I have recently noticed a couple flaky tests in the system-tests workflow.

[esecules]

Looks like the retry failed still 😔, however it's passing in my fork and my machine (MacBook).

Do these tests run in parallel and might interfere with each other?

Or the tests might be running on an underpowered server and the 60 second timeout isn't sufficient for GitHub Actions? Github's macos runners do only have 3 cores. Ubuntu runners have 4 cores. I am assuming apache/nifi uses the default runners.

[esecules]

Looks like system tests passed on retry! Thanks for all the retries.

[exceptionfactory]

Thanks for proposing these changes @esecules. I'm not complete sure about introducing the additional gauges because the processing performance information can be disabled. However, it looks like the null handling accounts for that fact, so this approach may be sufficient. I noted a few minor recommendations.

[exceptionfactory]

Instead of indicating the allowed values in the description, it should be possible to provide schema that indicates allowed values.

[exceptionfactory]

Although it is longer, including performance in the name would provide better alignment with the source of the information.

Suggested change

	.name("nifi_processing_cpu_duration")
	.name("nifi_processing_performance_cpu_duration")

[exceptionfactory]

Suggested change

	.help("Estimated cpu time (in milliseconds) used by this component")
	.help("Estimated CPU time (in milliseconds) used by this component")

[exceptionfactory]

Suggested change

	.help("Estimated gc time (in milliseconds) used by this component")
	.help("Estimated garbage collection time (in milliseconds) used by this component")

[esecules]

Thanks for proposing these changes @esecules. I'm not complete sure about introducing the additional gauges because the processing performance information can be disabled. However, it looks like the null handling accounts for that fact, so this approach may be sufficient. I noted a few minor recommendations.

I'll check if I already added test coverage for when the performance feature is disabled and I'll add it if it's missing.