Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add shellcheck to pre-commit and fix warnings #1427

Merged
merged 1 commit into from
Jan 22, 2025

Conversation

gforsyth
Copy link
Contributor

Description

shellcheck is a fast, static analysis tool for shell scripts. It's good at
flagging up unused variables, unintentional glob expansions, and other potential
execution and security headaches that arise from the wonders of bash (and other shlangs).

This PR adds a pre-commit hook to run shellcheck on all of the sh-lang files in the ci/ directory, and
the changes requested by shellcheck to make the existing files pass the check.

xref: rapidsai/build-planning#135

@gforsyth gforsyth requested a review from a team as a code owner January 22, 2025 14:48
@gforsyth gforsyth requested a review from bdice January 22, 2025 14:48
@github-actions github-actions bot added the ci label Jan 22, 2025
@gforsyth gforsyth added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change and removed ci labels Jan 22, 2025
Copy link
Member

@pentschev pentschev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks good, but there are a couple of entries I'm not sure about, I presume you have more experience with shellcheck so could you comment on those choices @gforsyth ?

Comment on lines +28 to +29
RAPIDS_DOCS_DIR="$(mktemp -d)"
export RAPIDS_DOCS_DIR
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks very weird to me and unnecessarily verbose, why is that change recommended by shellcheck?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bash footgun.
If it's a single-line, then the return code of the line is the result of export and it swallows the return code of the captured command.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And when running this locally, the errors will include a link explaining the issue: https://www.shellcheck.net/wiki/SC2155

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah makes sense, but it's also only problematic if you're running a command, if you're assigning a static value then exporting in the same line should have no problems. Thanks both!

@@ -9,10 +9,10 @@ rapids-logger "validate packages with 'pydistcheck'"

pydistcheck \
--inspect \
"$(echo ${wheel_dir_relative_path}/*.whl)"
"$(echo "${wheel_dir_relative_path}"/*.whl)"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe I'm mistaken, but wouldn't this mean ${wheel_dir_relative_path} is not within double-quotes since the open double-quote starts before $(echo ...?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another bash footgun, quoting resets inside of captures


rapids-logger "validate packages with 'twine'"

twine check \
--strict \
"$(echo ${wheel_dir_relative_path}/*.whl)"
"$(echo "${wheel_dir_relative_path}"/*.whl)"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to the above, doesn't this mean it's actually outside double-quotes?

Copy link
Member

@pentschev pentschev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @gforsyth .

@pentschev
Copy link
Member

/merge

@rapids-bot rapids-bot bot merged commit 4be4b9a into rapidsai:branch-25.02 Jan 22, 2025
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci improvement Improvement / enhancement to an existing function non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants