Cherry-pick Add gp_stat_progress_dtx_recovery for observability #872
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The "version" argument to ArgumentParser is deprecated. Please refer to https://bugs.launchpad.net/yade/+bug/1134422 Take a short example: ``` import argparse parser = argparse.ArgumentParser(version='1.0') ``` the error result is shown as below: ``` $ python3 main.py Traceback (most recent call last): File "main.py", line 2, in <module> parser = argparse.ArgumentParser(version='1.0') TypeError: __init__() got an unexpected keyword argument 'version' ``` **Solution**: use `parser.add_argument()` to fix it. Signed-off-by: Yongtao Huang <[email protected]>
When an error occurs in pg_basebackup(), its detail can not be printed properly. Instead, the following error will be raised: ``` +ERROR: TypeError: can only concatenate str (not "bytes") to str +CONTEXT: Traceback (most recent call last): + PL/Python function "pg_basebackup", line 26, in <module> + results = str(e) + "\ncommand output: " + e.output +PL/Python function "pg_basebackup" ``` This problem is caused by that e.output is a bytes array and cannot contact str. The solution is to change e.output to e.output.decode().
…g_transform Objects in these two catalogs do not have pg_depend nor pg_shdepend entries. We should ignore the dependency check for them like a few others. Fix https://github.com/greenplum-db/gpdb/issues/12016
Reviewed-by: Soumyadeep Chakraborty <[email protected]>
Long log: The `distutils `module was deprecated in Python 3.10 and is scheduled for removal in Python 3.12. At the same time, it is also dead code. Signed-off-by: Yongtao Huang <[email protected]>
We might be hitting a race condition where we are suspending `after_orphaned_check` on before suspending on `before_orphaned_check`, which may cause the `gp_inject_fault_infinite` of `before_orphaned_check` to never get triggered. Reorder the fault injection to fix the issue. Co-authored-by: Huansong Fu <[email protected]>
Long log: Module threading.Thread.getName() is deprecated since python 3.10. Deprecated getter/setter API for [name](https://docs.python.org/3/library/threading.html#threading.Thread.name); use it directly as a property instead. Take a simple example in python 3.10: ``` import threading import time def run(n): print("task", n) time.sleep(1) print('1s') time.sleep(1) print('0s') time.sleep(1) if __name__ == '__main__': t1 = threading.Thread(target=run, args=("t1",)) t1.start() print(t1.getName()) print(t1.name) ``` The result is shown as below: ``` $ python main.py task t1 /hoem/gpadmin/main.py:15: DeprecationWarning: getName() is deprecated, get the name attribute instead print(t1.getName()) Thread-1 (run) Thread-1 (run) 1s 0s ``` Signed-off-by: Yongtao Huang <[email protected]>
dtx recovery may take time sometimes and will leave you in the dark with how far the recovery has progressed. `gp_stat_progress_dtx_recovery` is a view that shows progress of the current dtx recovery. There are 5 main phases (excluding initializing): 1. recovering commited distributed transactions 2. gathering in-doubt transactions 3. aborting in-doubt transactions 4. gathering in-doubt orphaned transactions 5. managing in-doubt orphaned transactions `gp_stat_progress_dtx_recovery` will be shown with column: phase recover_commited_dtx_total recover_commited_dtx_completed in_doubt_tx_total in_doubt_tx_in_progress in_doubt_tx_aborted The wording of "managing" is used instead of "aborting" for in-doubt orphaned transactions to make it slightly ambigious. There is a chance an in-doubt transaction is already an on going transaction and there will be no need to dispatch a dtx abort command.
This reverts commit 85d943910c74b48e606515fc18a7d4a99e197dbc. Bypass resgroup still will put proc into cgroup and use a small value of query memory so it will not impact system too much. Revert previous commit let other application like GPCC can work easily.
reshke
previously approved these changes
Jan 16, 2025
my-ship-it
previously approved these changes
Jan 16, 2025
tuhaihe
dismissed stale reviews from my-ship-it and reshke
The base branch was changed.
|
Hi @my-ship-it @reshke, sorry for the review dismissal. Now our PRs are facing some unreasonable problems; we cannot rebase and merge the PR into the main branch. I just found one workaround[1]: edit the target branch to another one, then change back to the main branch. I would like to see if it works. Sorry for bothering you guys again. |
tuhaihe
approved these changes
Jan 16, 2025
my-ship-it
approved these changes
Jan 16, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #ISSUE_Number
What does this PR do?
Type of Change
Breaking Changes
Test Plan
make installcheckmake -C src/test installcheck-cbdb-parallelImpact
Performance:
User-facing changes:
Dependencies:
Checklist
Additional Context
CI Skip Instructions