lightning parallel import cause region count grows

[lance6716]

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

use multiple lightning node to import

2. What did you expect to see? (Required)

the region count should be almost the same as there's only one lightning node

3. What did you see instead (Required)

the region count is #lightning times than the single lightning import

4. What is your TiDB version? (Required)

because lightning will call PD SetRegionLabelRule, and this API will split region on the request key range. And lightning uses actual data range as the key range. When there're multiple lightning node, each node has different key range. So each node will causes region split once

tidb/pkg/lightning/backend/local/local.go

Lines 1331 to 1343 in 6c9ee36

	if len(splitKeys) > 0 && local.PausePDSchedulerScope == config.PausePDSchedulerScopeTable {
	log.FromContext(ctx).Info("pause pd scheduler of table scope")
	subCtx, cancel := context.WithCancel(ctx)
	defer cancel()
	var startKey, endKey []byte
	if len(splitKeys[0]) > 0 {
	startKey = codec.EncodeBytes(nil, splitKeys[0])
	}
	if len(splitKeys[len(splitKeys)-1]) > 0 {
	endKey = codec.EncodeBytes(nil, splitKeys[len(splitKeys)-1])
	}
	done, err := pdutil.PauseSchedulersByKeyRange(subCtx, local.pdHTTPCli, startKey, endKey)

maybe consider to use the whole range of tableID, or tableID + record/ indexID in the request

[OliverS929]

/assign OliverS929