Skip to content

Commit

Permalink
Merge #139735
Browse files Browse the repository at this point in the history
139735: cli: improve debug zip transaction_contention_events query r=kyle-a-wong a=kyle-a-wong

Previously, this query often timed out with errors like "memory budget exceeded" and "query execution canceled due to statement timeout." This was happening because the query needed to deduplicate all the rows returned after performing two left joins on system.statement_statistics. This table is denormalized, meaning many rows are returned for every row joined.

To improve this query, CTEs are used to first create distinct (statement_fingerprint, query) and (transaction_fingerprint, queries) tables. Then, transaction_contention_events is joined onto those tables. This reduces the amount of deduplication required, improving performance.

Fixes: CRDB-45216
Epic: none
Release note (cli change): Improves the performance of the debug zip query that collects transaction_contention_events data, reducing the chances of "memory budget exceeded" or "query execution canceled due to statement timeout" errors.

Co-authored-by: Kyle Wong <[email protected]>
  • Loading branch information
craig[bot] and kyle-a-wong committed Jan 24, 2025
2 parents 5599daf + 10308f1 commit 2698603
Showing 1 changed file with 19 additions and 8 deletions.
27 changes: 19 additions & 8 deletions pkg/cli/zip_table_registry.go
Original file line number Diff line number Diff line change
Expand Up @@ -550,26 +550,37 @@ var zipInternalTablesPerCluster = DebugZipTableRegistry{
},
"crdb_internal.transaction_contention_events": {
customQueryUnredacted: `
with fingerprint_queries as (
SELECT distinct fingerprint_id, metadata->> 'query' as query
FROM system.statement_statistics
),
transaction_fingerprints as (
SELECT distinct fingerprint_id, transaction_fingerprint_id
FROM system.statement_statistics
),
transaction_queries as (
SELECT tf.transaction_fingerprint_id, array_agg(fq.query) as queries
FROM fingerprint_queries fq
JOIN transaction_fingerprints tf on tf.fingerprint_id = fq.fingerprint_id
GROUP BY tf.transaction_fingerprint_id
)
SELECT collection_ts,
contention_duration,
waiting_txn_id,
waiting_txn_fingerprint_id,
waiting_stmt_fingerprint_id,
s.metadata ->> 'query' AS waiting_stmt_query,
fq.query AS waiting_stmt_query,
blocking_txn_id,
blocking_txn_fingerprint_id,
array_agg(distinct ss.metadata ->> 'query') AS blocking_txn_queries_unordered,
tq.queries AS blocking_txn_queries_unordered,
contending_pretty_key,
index_name,
table_name,
database_name
FROM crdb_internal.transaction_contention_events
LEFT JOIN system.statement_statistics AS s ON waiting_stmt_fingerprint_id = s.fingerprint_id
LEFT JOIN system.statement_statistics AS ss ON ss.transaction_fingerprint_id = blocking_txn_fingerprint_id
WHERE ss.transaction_fingerprint_id != '\x0000000000000000' AND s.fingerprint_id != '\x0000000000000000'
GROUP BY collection_ts, contention_duration, waiting_txn_id, waiting_txn_fingerprint_id, blocking_txn_id,
blocking_txn_fingerprint_id, waiting_stmt_fingerprint_id, contending_pretty_key, s.metadata ->> 'query',
index_name, table_name, database_name
LEFT JOIN fingerprint_queries fq ON fq.fingerprint_id = waiting_stmt_fingerprint_id
LEFT JOIN transaction_queries tq ON tq.transaction_fingerprint_id = blocking_txn_fingerprint_id
WHERE fq.fingerprint_id != '\x0000000000000000' AND tq.transaction_fingerprint_id != '\x0000000000000000'
`,
customQueryUnredactedFallback: `
SELECT collection_ts,
Expand Down

0 comments on commit 2698603

Please sign in to comment.