HBASE-29084: Track table level metrics on HBase client side

[sanjeet006py]

Summary of the change:

• Introduced new class DelegateTableRunnableWithMetrics to wrap a runnable and track metrics like execution time and queue wait time. Wrapped existing runnable used by HTable#batch() API within DelegateTableRunnableWithMetrics.
• Introduced class TableThreadMetricsHolder to hold metrics for a runnable. There is one-to-one mapping b/w instance of DelegateTableRunnableWithMetrics and TableThreadMetricsHolder.
• Introduced class TableMetrics to collect TableThreadMetricsHolder objects. There is one TableMetrics object per HTable object.
• Exposes list of TableThreadMetricsHolder objects by clients so, that clients can aggregate the metrics as per their requirement. Thus, giving them flexibility. But have provided one basic implementation of aggregating metrics from TableThreadMetricsHolder objects list.

Looking for inputs/suggestions around:

• Overall design of the changes.
• TableMetrics uses synchronized list to store TableThreadMetricsHolder objects. Understanding is that its very unlikely that multiple threads are trying to add TableThreadMetricsHolder objects to the list simultaneously. But wondering if this can cause batch calls to see increase in latency and any better alternatives?
• Introduced a config hbase.client.table.metrics.enable to skip wrapping runnable inside DelegateTableRunnableWithMetrics. Not sure why anyone would want to skip wrapping the runnable given whether to consume metrics is totally upto end user. Maybe we can remove config?

Pending:

• Adding test coverage.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 47s		Docker mode activated.
-0 ⚠️	yetus	0m 6s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2.6 Compile Tests _
+0 🆗	mvndep	0m 16s		Maven dependency ordering for branch
+1 💚	mvninstall	2m 49s		branch-2.6 passed
+1 💚	compile	0m 36s		branch-2.6 passed
+1 💚	javadoc	0m 34s		branch-2.6 passed
+1 💚	shadedjars	5m 30s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 58s		the patch passed
+1 💚	compile	0m 37s		the patch passed
+1 💚	javac	0m 37s		the patch passed
+1 💚	javadoc	0m 31s		the patch passed
+1 💚	shadedjars	5m 44s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	2m 9s		hbase-common in the patch passed.
+1 💚	unit	8m 7s		hbase-client in the patch passed.
		32m 43s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6603/1/artifact/yetus-jdk8-hadoop2-check/output/Dockerfile
GITHUB PR	#6603
JIRA Issue	HBASE-29084
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 91b21ee5cf5e 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.6 / 361b556
Default Java	Temurin-1.8.0_412-b08
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6603/1/testReport/
Max. process+thread count	350 (vs. ulimit of 30000)
modules	C: hbase-common hbase-client U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6603/1/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 43s		Docker mode activated.
-0 ⚠️	yetus	0m 6s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2.6 Compile Tests _
+0 🆗	mvndep	0m 11s		Maven dependency ordering for branch
+1 💚	mvninstall	3m 3s		branch-2.6 passed
+1 💚	compile	0m 39s		branch-2.6 passed
+1 💚	javadoc	0m 33s		branch-2.6 passed
+1 💚	shadedjars	5m 52s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 16s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 57s		the patch passed
+1 💚	compile	0m 40s		the patch passed
+1 💚	javac	0m 40s		the patch passed
+1 💚	javadoc	0m 32s		the patch passed
+1 💚	shadedjars	5m 48s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	2m 7s		hbase-common in the patch passed.
+1 💚	unit	8m 3s		hbase-client in the patch passed.
		33m 4s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6603/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#6603
JIRA Issue	HBASE-29084
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 712135ef515e 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.6 / 361b556
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6603/1/testReport/
Max. process+thread count	382 (vs. ulimit of 30000)
modules	C: hbase-common hbase-client U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6603/1/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 48s		Docker mode activated.
-0 ⚠️	yetus	0m 6s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ branch-2.6 Compile Tests _
+0 🆗	mvndep	0m 12s		Maven dependency ordering for branch
+1 💚	mvninstall	3m 16s		branch-2.6 passed
+1 💚	compile	0m 38s		branch-2.6 passed
+1 💚	javadoc	0m 35s		branch-2.6 passed
+1 💚	shadedjars	6m 24s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 17s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 10s		the patch passed
+1 💚	compile	0m 38s		the patch passed
+1 💚	javac	0m 38s		the patch passed
+1 💚	javadoc	0m 33s		the patch passed
+1 💚	shadedjars	6m 21s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	2m 20s		hbase-common in the patch passed.
+1 💚	unit	8m 19s		hbase-client in the patch passed.
		35m 12s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6603/1/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile
GITHUB PR	#6603
JIRA Issue	HBASE-29084
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux cc7019e5e8c7 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.6 / 361b556
Default Java	Eclipse Adoptium-11.0.23+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6603/1/testReport/
Max. process+thread count	366 (vs. ulimit of 30000)
modules	C: hbase-common hbase-client U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6603/1/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 49s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ branch-2.6 Compile Tests _
+0 🆗	mvndep	0m 12s		Maven dependency ordering for branch
+1 💚	mvninstall	3m 47s		branch-2.6 passed
+1 💚	compile	1m 54s		branch-2.6 passed
+1 💚	checkstyle	0m 39s		branch-2.6 passed
+1 💚	spotbugs	1m 58s		branch-2.6 passed
+1 💚	spotless	0m 56s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 17s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 58s		the patch passed
+1 💚	compile	2m 15s		the patch passed
+1 💚	javac	2m 15s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	0m 22s	/results-checkstyle-hbase-common.txt	hbase-common: The patch generated 1 new + 8 unchanged - 0 fixed = 9 total (was 8)
-0 ⚠️	checkstyle	0m 29s	/results-checkstyle-hbase-client.txt	hbase-client: The patch generated 6 new + 13 unchanged - 0 fixed = 19 total (was 13)
+1 💚	xmllint	0m 0s		No new issues.
+1 💚	spotbugs	2m 17s		the patch passed
+1 💚	hadoopcheck	18m 24s		Patch does not cause any errors with Hadoop 2.10.2 or 3.3.6 3.4.0.
-1 ❌	spotless	0m 17s		patch has 28 errors when running spotless:check, run spotless:apply to fix.
			_ Other Tests _
+1 💚	asflicense	0m 20s		The patch does not generate ASF License warnings.
		41m 10s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6603/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#6603
JIRA Issue	HBASE-29084
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless xmllint
uname	Linux 5a95ce484ef6 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	branch-2.6 / 361b556
Default Java	Eclipse Adoptium-11.0.23+9
spotless	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6603/1/artifact/yetus-general-check/output/patch-spotless.txt
Max. process+thread count	78 (vs. ulimit of 30000)
modules	C: hbase-common hbase-client U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-6603/1/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3 xmllint=20913
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[apurtell]

Have you looked at MetricsConnection? e.g.

• https://github.com/apache/hbase/blob/branch-2/hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetricsConnection.java
• https://github.com/apache/hbase/blob/master/hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetricsConnection.java#L97
• https://github.com/apache/hbase/blob/branch-2/hbase-client/src/main/java/org/apache/hadoop/hbase/client/MetricsConnection.java#L829