Consider removing elasticsearch_shards and elasticsearch_replicas server configuration settings

[danotorrey]

What?

Consider removing the elasticsearch_shards and elasticsearch_replicas server configuration settings (possibly in Graylog 5.0).

#13018 establishes an in-database method of storing these values (which can be changed at runtime through the System > Configurations page and through the Graylog Cluster Config API). When values are not set in the database, then the existing server config settings are used.

Also see https://github.com/Graylog2/graylog-plugin-enterprise/issues/3319.

Why?

The aforementioned server config settings are deprecated and can probably be removed at a sensible time (possibly in Graylog 5.0) and instead rely on the new in-database values.

[danotorrey]

Note: These configuration settings are not used in the new index creation flow, and are only used in some older migrations.

The defaults for the new index flow are currently hard-coded in the frontend:

graylog2-server/graylog2-web-interface/src/pages/IndexSetCreationPage.tsx

Lines 126 to 127 in 85eb375

	shards: 4,
	replicas: 0,

So, this probably makes it even easier to remove these config values, since not much is relying on them.

[danotorrey]

We have since identified that these properties are needed for the initialization of index sets (and the default) for newly created installations (in the case that it is desired to initialize the properties to a specific value on first boot). Therefore, they cannot be removed.

See #13018 for more info.

[williamtrelawny]

Sorry to necro this thread, but I just had a customer who installed a net new 6.1 Graylog and encountered this issue. They wanted to set the default shard count for their Opensearch and his first instinct was to go to the config file. He changed these values but noticed there was no effect. He finally managed to find the new "Index Set Templates" UI menu which did the trick.

I see above that we thought 5.0 would be a sensible time to remove these fields entirely, and I see they are commented out by default. I think removing them completely is long past due, unless there are any remaining technical limitations? Can't imagine so since these values are commented out, right?

[T100D]

These values are normally needed at the first installation and are then ignored on further installations, but are essential to set elasticsearch by default.

It could be removed of it is by default present at first installation.