[INTERNAL] Allow intra-broker goals to be used as default

[alexander-falca]

This PR resolves linkedin#1264

It seems that intra.broker.goals setting is checked at config validation only and this PR updates sanity check logic.

Adding WIP label while testing in dev env.

[amuraru]

@alexander-falca I am not sure if this is enough to implement linkedin#1264

The need for us is to have the intra-broker goals available in self.healer. Meaning that similarly to the other inter brokers goals we'd need the healer to continously analyze these intra-broker goals and self-heal them automatically.
CC maintainers suggested we could have a second healer - one dedicated for these goals - that can run in parallel with the current one.

[ecojan]

Small things to mention:

Building the image via docker build should be stated to be started from the root path of the project using since the dockerfile is not residing in the root folder:

export TAG=v1.0.0-sometag
docker build -t cc-image:$TAG -f ./docker/Dockerfile .

• topic.config.provider has changed default value and com.linkedin.kafka.cruisecontrol.config.KafkaAdminTopicConfigProvider does not exist anymore
• min.topic.leaders.per.broker has a minimum of 1 now (-1 is not allowed)

Goals seem to be added successfully to the list of goals for self-healing, in the UI, the proposals tab has a generic table that doesn't do any justice to the intra-broker goals.

IntraBrokerDiskUsageDistributionGoal is also requiring 20 snapshots by default (so partition movements via this goal will wait a lot of something changes in the cluster, eg. replica movement)

[ecojan]

Found the following error in logs

[2021-10-07 09:27:23,902] ERROR Uncaught exception in anomaly handler. (com.linkedin.kafka.cruisecontrol.detector.AnomalyDetectorManager)
com.linkedin.kafka.cruisecontrol.exception.KafkaCruiseControlException: java.lang.IllegalArgumentException: IntraBrokerDiskCapacityGoal does not support balancing action not specifying logdir.
	at com.linkedin.kafka.cruisecontrol.servlet.handler.async.runnable.GoalBasedOperationRunnable.computeResult(GoalBasedOperationRunnable.java:167) ~[cruise-control.jar:?]
	at com.linkedin.kafka.cruisecontrol.servlet.handler.async.runnable.RebalanceRunnable.workWithoutClusterModel(RebalanceRunnable.java:115) ~[cruise-control.jar:?]
	at com.linkedin.kafka.cruisecontrol.servlet.handler.async.runnable.GoalBasedOperationRunnable.computeResult(GoalBasedOperationRunnable.java:173) ~[cruise-control.jar:?]
	at com.linkedin.kafka.cruisecontrol.detector.GoalViolations.fix(GoalViolations.java:89) ~[cruise-control.jar:?]
	at com.linkedin.kafka.cruisecontrol.detector.AnomalyDetectorManager$AnomalyHandlerTask.fixAnomalyInProgress(AnomalyDetectorManager.java:547) ~[cruise-control.jar:?]
	at com.linkedin.kafka.cruisecontrol.detector.AnomalyDetectorManager$AnomalyHandlerTask.processAnomalyInProgress(AnomalyDetectorManager.java:413) ~[cruise-control.jar:?]
	at com.linkedin.kafka.cruisecontrol.detector.AnomalyDetectorManager$AnomalyHandlerTask.handleAnomalyInProgress(AnomalyDetectorManager.java:396) ~[cruise-control.jar:?]
	at com.linkedin.kafka.cruisecontrol.detector.AnomalyDetectorManager$AnomalyHandlerTask.run(AnomalyDetectorManager.java:358) [cruise-control.jar:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]
	at java.lang.Thread.run(Thread.java:831) [?:?]
Caused by: java.lang.IllegalArgumentException: IntraBrokerDiskCapacityGoal does not support balancing action not specifying logdir.
	at com.linkedin.kafka.cruisecontrol.analyzer.goals.IntraBrokerDiskCapacityGoal.actionAcceptance(IntraBrokerDiskCapacityGoal.java:129) ~[cruise-control.jar:?]
	at com.linkedin.kafka.cruisecontrol.analyzer.AnalyzerUtils.isProposalAcceptableForOptimizedGoals(AnalyzerUtils.java:121) ~[cruise-control.jar:?]
	at com.linkedin.kafka.cruisecontrol.analyzer.goals.AbstractGoal.maybeApplyBalancingAction(AbstractGoal.java:254) ~[cruise-control.jar:?]
	at com.linkedin.kafka.cruisecontrol.analyzer.goals.MinTopicLeadersPerBrokerGoal.maybeMoveLeaderOfTopicToBroker(MinTopicLeadersPerBrokerGoal.java:340) ~[cruise-control.jar:?]
	at com.linkedin.kafka.cruisecontrol.analyzer.goals.MinTopicLeadersPerBrokerGoal.rebalanceForBroker(MinTopicLeadersPerBrokerGoal.java:316) ~[cruise-control.jar:?]
	at com.linkedin.kafka.cruisecontrol.analyzer.goals.AbstractGoal.optimize(AbstractGoal.java:100) ~[cruise-control.jar:?]
	at com.linkedin.kafka.cruisecontrol.analyzer.GoalOptimizer.optimizations(GoalOptimizer.java:447) ~[cruise-control.jar:?]
	at com.linkedin.kafka.cruisecontrol.KafkaCruiseControl.optimizations(KafkaCruiseControl.java:583) ~[cruise-control.jar:?]
	at com.linkedin.kafka.cruisecontrol.servlet.handler.async.runnable.ProposalsRunnable.workWithClusterModel(ProposalsRunnable.java:114) ~[cruise-control.jar:?]
	at com.linkedin.kafka.cruisecontrol.servlet.handler.async.runnable.GoalBasedOperationRunnable.computeResult(GoalBasedOperationRunnable.java:161) ~[cruise-control.jar:?]
	... 13 more

[ecojan]

Does the same for "IntraBrokerDiskUsageDistributionGoal" as well.

[alexander-falca]

Closing in favour of linkedin#1721