-
Notifications
You must be signed in to change notification settings - Fork 5
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add lock-files script #17
Open
tcharding
wants to merge
1
commit into
rust-bitcoin:master
Choose a base branch
from
tcharding:09-20-lock-files
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+364
−0
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,203 @@ | ||
#!/usr/bin/env bash | ||
# | ||
# Create and manipulate the lock files. | ||
# | ||
# We use lock files `Cargo-recent.lock` and `Cargo-minimal.lock` to pin | ||
# dependencies and check specific versions in CI. | ||
# | ||
# Shellcheck can't search dynamic paths | ||
# shellcheck source=/dev/null | ||
|
||
set -u | ||
|
||
main() { | ||
set_globals | ||
assert_cmds | ||
handle_command_line_args "$@" | ||
} | ||
|
||
set_globals() { | ||
# The directory of this script. | ||
script_dir=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) | ||
|
||
# Used below by `say`. | ||
script=${0##*/} | ||
|
||
# Used below by `verbose_say`. | ||
flag_verbose=false | ||
|
||
# Sourcing this file requires `flag_verbose` to be set. | ||
. "$script_dir/stdlib.sh" | ||
|
||
# Environment sanity checks | ||
assert_nz "$HOME" "\$HOME is undefined" | ||
assert_nz "$0" "\$0 is undefined" | ||
|
||
# Make all cargo invocations verbose. | ||
export CARGO_TERM_VERBOSE=true | ||
|
||
recent="Cargo-recent.lock" | ||
minimal="Cargo-minimal.lock" | ||
|
||
msrv="1.63.0" | ||
} | ||
|
||
handle_command_line_args() { | ||
local _no_args=false | ||
local _help=false | ||
local _create=false | ||
local _msrv="" | ||
local _update=false | ||
|
||
if [ $# -eq 0 ]; then | ||
_no_args=true | ||
fi | ||
|
||
local _arg | ||
for _arg in "$@"; do | ||
case "${_arg%%=*}" in | ||
create ) | ||
_create=true | ||
;; | ||
|
||
update ) | ||
_update=true | ||
;; | ||
|
||
--msrv ) | ||
if is_value_arg "$_arg" "--msrv"; then | ||
_msrv="$(get_value_arg "$_arg")" | ||
else | ||
say_err "the --msrv option requires a toolchain version argument" | ||
print_help | ||
exit 1 | ||
fi | ||
|
||
;; | ||
|
||
-h | --help ) | ||
_help=true | ||
;; | ||
|
||
--verbose) | ||
# verbose is a global flag | ||
flag_verbose=true | ||
;; | ||
|
||
*) | ||
echo "Unknown argument '$_arg', displaying usage:" | ||
echo "${_arg%%=*}" | ||
_help=true | ||
;; | ||
|
||
esac | ||
|
||
done | ||
|
||
if [ "$_create" = true ]; then | ||
if [ -z "$_msrv" ]; then | ||
msrv="$_msrv" | ||
fi | ||
|
||
verbose_say "Creating lock files, MSRV: $msrv" | ||
create | ||
fi | ||
|
||
if [ "$_update" = true ]; then | ||
update | ||
verbose_say "Your git index will now be dirty if lock file update is required" | ||
fi | ||
|
||
if [ "$_help" = true ]; then | ||
print_help | ||
exit 0 | ||
fi | ||
|
||
if [ "$_no_args" = true ]; then | ||
verbose_say "no option supplied, defaulting to update" | ||
update | ||
fi | ||
} | ||
|
||
is_value_arg() { | ||
local _arg="$1" | ||
local _name="$2" | ||
|
||
echo "$_arg" | grep -q -- "$_name=" | ||
return $? | ||
} | ||
|
||
get_value_arg() { | ||
local _arg="$1" | ||
|
||
echo "$_arg" | cut -f2 -d= | ||
} | ||
|
||
# Creates the minimal and recent lock files. | ||
# | ||
# If this function fails you may want to be lazy and just duplicate `Cargo.lock` | ||
# as the minimal and recent lock files. | ||
create() { | ||
# Attempt to create a minimal lock file. | ||
rm --force Cargo.lock > /dev/null | ||
cargo +nightly check --all-features -Z minimal-versions | ||
need_ok "failed to build with -Z minimial-versions, you might have to use a recent lock file for minimal" | ||
|
||
# We only want to create the minimal lock file if we can build with current MSRV. | ||
cargo "+$msrv" --locked check --all-features | ||
need_ok "failed to build with minimal lock file and MSRV $_msrv" | ||
cp Cargo.lock "$minimal" | ||
|
||
# If that worked we can create a recent lock file. | ||
cargo update | ||
cp Cargo.lock "$recent" | ||
} | ||
|
||
# Updates the minimal and recent lock files. | ||
update() { | ||
for file in "$minimal" "$recent"; do | ||
cp --force "$file" Cargo.lock | ||
cargo check | ||
cp --force Cargo.lock "$file" | ||
done | ||
} | ||
|
||
assert_cmds() { | ||
need_cmd cat | ||
need_cmd cp | ||
need_cmd cargo | ||
} | ||
|
||
say() { | ||
echo "$script: $1" | ||
} | ||
|
||
say_err() { | ||
say "$1" >&2 | ||
} | ||
|
||
verbose_say() { | ||
if [ "$flag_verbose" = true ]; then | ||
say "$1" | ||
fi | ||
} | ||
|
||
print_help() { | ||
cat <<EOF | ||
Usage: $script [OPITON] [COMMAND] | ||
|
||
COMMAND: | ||
|
||
create Create new minimal and recent lock files. | ||
update Update the minimal and recent lock files (default). | ||
|
||
OPTION: | ||
|
||
--msrv=version The Rust toolchain version to use as MSRV. | ||
--verbose Enable verbose output. | ||
--help, -h Display this help information. | ||
EOF | ||
} | ||
|
||
# Main script | ||
main "$@" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,56 @@ | ||
# rust-bitcoin stdlib for Bash (of sorts). | ||
# | ||
# Much of the code here was originally stolen from the old `rustup.sh` script | ||
# https://github.com/rust-lang-deprecated/rustup.sh/blob/master/rustup.sh | ||
# | ||
# If you have never read the comments at the top of that file, consider it, they are gold. | ||
# | ||
# No shebang, this file should not be executed. | ||
# shellcheck disable=SC2148 | ||
# | ||
# Disable because `flag_verbose` is referenced but not assigned, however we check it is non-zero. | ||
# shellcheck disable=SC2154 | ||
|
||
set -u | ||
|
||
err() { | ||
echo "$1" >&2 | ||
exit 1 | ||
} | ||
|
||
need_cmd() { | ||
if ! command -v "$1" > /dev/null 2>&1 | ||
then err "need '$1' (command not found)" | ||
fi | ||
} | ||
|
||
need_ok() { | ||
if [ $? != 0 ]; then err "$1"; fi | ||
} | ||
|
||
assert_nz() { | ||
if [ -z "$1" ]; then err "assert_nz $2"; fi | ||
} | ||
|
||
# Run a command that should never fail. If the command fails execution | ||
# will immediately terminate with an error showing the failing | ||
# command. | ||
ensure() { | ||
"$@" | ||
need_ok "command failed: $*" | ||
} | ||
|
||
# This is just for indicating that commands' results are being | ||
# intentionally ignored. Usually, because it's being executed | ||
# as part of error handling. | ||
ignore() { | ||
run "$@" | ||
} | ||
|
||
# Assert that we have a nightly Rust toolchain installed. | ||
need_nightly() { | ||
cargo_ver=$(cargo --version) | ||
if echo "$cargo_ver" | grep -q -v nightly; then | ||
err "Need a nightly compiler; have $(cargo --version) (use RUSTUP_TOOLCHAIN=+nightly cmd)" | ||
fi | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,105 @@ | ||
#!/usr/bin/env bash | ||
# | ||
# Bash shell script template. | ||
# | ||
# Shellcheck can't search dynamic paths | ||
# shellcheck source=/dev/null | ||
|
||
# Note we don't use `set -x` because error handling is done manually. | ||
# If you use pipes you may want to use `set -euxo pipefail` instead. | ||
set -u | ||
|
||
main() { | ||
set_globals | ||
assert_cmds | ||
handle_command_line_args "$@" | ||
} | ||
|
||
set_globals() { | ||
# The git repository where script is run from. | ||
# repo_dir=$(git rev-parse --show-toplevel) | ||
|
||
# The directory of this script. | ||
script_dir=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) | ||
|
||
# Used below by `say`. | ||
script=${0##*/} | ||
|
||
# Used below by `verbose_say`. | ||
flag_verbose=false | ||
|
||
# Sourcing this file requires `flag_verbose` to be set. | ||
. "$script_dir/stdlib.sh" | ||
|
||
# Environment sanity checks | ||
assert_nz "$HOME" "\$HOME is undefined" | ||
assert_nz "$0" "\$0 is undefined" | ||
|
||
# Make all cargo invocations verbose. | ||
export CARGO_TERM_VERBOSE=true | ||
} | ||
|
||
handle_command_line_args() { | ||
local _help=false | ||
|
||
local _arg | ||
for _arg in "$@"; do | ||
case "${_arg%%=*}" in | ||
-h | --help ) | ||
_help=true | ||
;; | ||
|
||
--verbose) | ||
# verbose is a global flag | ||
flag_verbose=true | ||
;; | ||
|
||
*) | ||
echo "Unknown argument '$_arg', displaying usage:" | ||
echo "${_arg%%=*}" | ||
_help=true | ||
;; | ||
|
||
esac | ||
|
||
done | ||
|
||
if [ "$_help" = true ]; then | ||
print_help | ||
exit 0 | ||
fi | ||
|
||
verbose_say "Enabled verbose output" | ||
} | ||
|
||
assert_cmds() { | ||
need_cmd cat | ||
} | ||
|
||
say() { | ||
echo "$script: $1" | ||
} | ||
|
||
say_err() { | ||
say "$1" >&2 | ||
} | ||
|
||
verbose_say() { | ||
if [ "$flag_verbose" = true ]; then | ||
say "$1" | ||
fi | ||
} | ||
|
||
print_help() { | ||
cat <<EOF | ||
Usage: $script [--verbose] | ||
|
||
Options: | ||
|
||
--verbose, Enable verbose output | ||
--help, -h Display usage information | ||
EOF | ||
} | ||
|
||
# Main script | ||
main "$@" |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In 65a74a1:
It seems like this script needs this line to fail in order to be useful. But I believe this fails on every single one of our repos becaue our minimal dependencies themselves don't specify their minimal dependencies correctly.
Though I guess I haven't tried it for some of the new leaf crates..
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What do you mean by this? My main idea behind this script was that if this succeeds then win and one doesn't have to have the
-Z
command in shell history or in mind. And if it this fails the script documents my lazy 'just duplicate recent'. But since the script lives in an obscure repo is this even useful, I don't know?I used
-Z minimal-versions
recently and it worked, to my surprise (maybe inbech32
, but not sure).There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ooo, nice.
I think the MSRV update to 1.63 solved a lot of our dependency compatibility issues.
I don't think it's too useful as documentation because it's inside a script in a fairly obscure repo.
I also think we should move away from "duplicate the recent lockfile" to "just don't have a minimal lockfile". This will clearly signal which repos still need somebody to do the hard work of running
-Z minimal
then patching everything up.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess, if the script were to try using -Zminimal, and on failure, output a warning and not produce the lockfile, that'd be pretty cool.
I guess we also need to check that the "recent" lockfile produced by
cargo update
works with MSRV. Again, in my experience it almost never does. And we need to hard fail if neither lockfile can be produced.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good ideas, will re-spin.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently the script doesn't fail correctly in workspace because
cargo check
in a workspace does not exit with non-zero if the last crate builds successfully, I couldn't find a way to make it do so.Leaving this for another day.