Disabling node level metrics collection issue

[mashhurs]

Description of the problem including expected versus actual behavior

Logstash has node and pipeline level metric collection control signal. As of now (after #16864 fix), users will be able to disable pipeline level metric collection. However, when node level metric collection disable is impossible which leads endpoint failure.
Extra notes: not to confuse with api.enabled which disables the LS web server, it has broader context than metrics that loggin CRUD operations can be proceed.

Acceptance Criteria

• Fix the current bug;
• End users should have a meaningful guidance (reasoning and guide to enable back) when metric collection disabled and hit the endpoints
• Documentation to highlight metric collection enable/disable

Steps to reproduce:

• Use Logstash built from main or 8.17.0
• Set metric.collect: false in logstash.yml
• Run Logstash
• Hit the localhost:9600/_node/stats endpoint

which results error in both LS console/log and endpoint result.

Provide logs

# Logstash log
[2025-01-09T12:51:02,844][ERROR][logstash.agent           ] Internal API server error {:status=>500, :request_method=>"GET", :path_info=>"/_node/stats/flow", :query_string=>"", :http_version=>"HTTP/1.1", :http_accept=>"*/*", :error=>"Unexpected Internal Error", :class=>"LogStash::Instrument::MetricStore::MetricNotFound", :message=>"For path: jvm. Map keys: []", :backtrace=>["/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:249:in `block in get_recursively'", "org/jruby/RubyArray.java:1981:in `each'", "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:248:in `get_recursively'", "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:120:in `block in get'", "org/jruby/ext/thread/Mutex.java:174:in `synchronize'", "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:119:in `get'", "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:133:in `get_shallow'", "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:181:in `block in extract_metrics'", "org/jruby/RubyArray.java:1981:in `each'", "org/jruby/RubyEnumerable.java:1084:in `inject'", "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:157:in `extract_metrics'", "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/service.rb:45:in `extract_metrics'", "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/commands/base.rb:37:in `extract_metrics'", "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/commands/stats.rb:52:in `jvm'", "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/modules/node_stats.rb:70:in `jvm_payload'", "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/modules/node_stats.rb:34:in `block in NodeStats'", "org/jruby/RubyMethod.java:119:in `call'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1807:in `block in compile!'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1074:in `block in route!'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1092:in `route_eval'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1074:in `block in route!'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1123:in `block in process_route'", "org/jruby/RubyKernel.java:1426:in `catch'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1121:in `process_route'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1072:in `block in route!'", "org/jruby/RubyArray.java:1981:in `each'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1069:in `route!'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1193:in `block in dispatch!'", "org/jruby/RubyKernel.java:1426:in `catch'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1164:in `invoke'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1188:in `dispatch!'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1004:in `block in call!'", "org/jruby/RubyKernel.java:1426:in `catch'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1164:in `invoke'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1004:in `call!'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:993:in `call'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/base.rb:53:in `call'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/xss_header.rb:20:in `call'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/path_traversal.rb:18:in `call'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/json_csrf.rb:28:in `call'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/base.rb:53:in `call'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/base.rb:53:in `call'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/frame_options.rb:33:in `call'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/middleware/logger.rb:17:in `call'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-3.1.8/lib/rack/head.rb:15:in `call'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:227:in `call'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:2138:in `call'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-3.1.8/lib/rack/urlmap.rb:76:in `block in call'", "org/jruby/RubyArray.java:1981:in `each'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-3.1.8/lib/rack/urlmap.rb:60:in `call'", "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/rack_app.rb:75:in `call'", "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/rack_app.rb:49:in `call'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-3.1.8/lib/rack/builder.rb:277:in `call'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/request.rb:99:in `block in handle_request'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/thread_pool.rb:389:in `with_force_shutdown'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/request.rb:98:in `handle_request'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/server.rb:468:in `process_client'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/server.rb:249:in `block in run'", "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/thread_pool.rb:166:in `block in spawn_thread'"]}
[2025-01-09T12:51:02,863][ERROR][logstash.agent           ] API HTTP Request {:status=>500, :request_method=>"GET", :path_info=>"/_node/stats/flow", :query_string=>"", :http_version=>"HTTP/1.1", :http_accept=>"*/*"}


# API call response
{
    "status": 500,
    "request_method": "GET",
    "path_info": "/_node/stats",
    "query_string": "",
    "http_version": "HTTP/1.1",
    "http_accept": "*/*",
    "error": "Unexpected Internal Error",
    "class": "LogStash::Instrument::MetricStore::MetricNotFound",
    "message": "For path: jvm. Map keys: []",
    "backtrace": [
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:249:in `block in get_recursively'",
        "org/jruby/RubyArray.java:1981:in `each'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:248:in `get_recursively'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:120:in `block in get'",
        "org/jruby/ext/thread/Mutex.java:174:in `synchronize'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:119:in `get'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:133:in `get_shallow'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:181:in `block in extract_metrics'",
        "org/jruby/RubyArray.java:1981:in `each'",
        "org/jruby/RubyEnumerable.java:1084:in `inject'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/instrument/metric_store.rb:157:in `extract_metrics'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/service.rb:45:in `extract_metrics'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/commands/base.rb:37:in `extract_metrics'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/commands/stats.rb:52:in `jvm'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/modules/node_stats.rb:70:in `jvm_payload'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/modules/node_stats.rb:34:in `block in NodeStats'",
        "org/jruby/RubyMethod.java:119:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1807:in `block in compile!'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1074:in `block in route!'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1092:in `route_eval'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1074:in `block in route!'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1123:in `block in process_route'",
        "org/jruby/RubyKernel.java:1426:in `catch'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1121:in `process_route'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1072:in `block in route!'",
        "org/jruby/RubyArray.java:1981:in `each'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1069:in `route!'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1193:in `block in dispatch!'",
        "org/jruby/RubyKernel.java:1426:in `catch'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1164:in `invoke'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1188:in `dispatch!'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1004:in `block in call!'",
        "org/jruby/RubyKernel.java:1426:in `catch'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1164:in `invoke'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:1004:in `call!'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:993:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/base.rb:53:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/xss_header.rb:20:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/path_traversal.rb:18:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/json_csrf.rb:28:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/base.rb:53:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/base.rb:53:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-protection-4.1.1/lib/rack/protection/frame_options.rb:33:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/middleware/logger.rb:17:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-3.1.8/lib/rack/head.rb:15:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:227:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/sinatra-4.1.1/lib/sinatra/base.rb:2138:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-3.1.8/lib/rack/urlmap.rb:76:in `block in call'",
        "org/jruby/RubyArray.java:1981:in `each'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-3.1.8/lib/rack/urlmap.rb:60:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/rack_app.rb:75:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/logstash-core/lib/logstash/api/rack_app.rb:49:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/rack-3.1.8/lib/rack/builder.rb:277:in `call'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/request.rb:99:in `block in handle_request'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/thread_pool.rb:389:in `with_force_shutdown'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/request.rb:98:in `handle_request'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/server.rb:468:in `process_client'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/server.rb:249:in `block in run'",
        "/Users/mashhur/Dev/elastic/logstash/vendor/bundle/jruby/3.1.0/gems/puma-6.5.0-java/lib/puma/thread_pool.rb:166:in `block in spawn_thread'"
    ]
}