-
Notifications
You must be signed in to change notification settings - Fork 14
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
add CI workflows running tests (#15)
Closes rapidsai/graph_dl#638 Adds CI workflows running the unit tests. Also adds some `.pre-commit-config.yaml` changes, follow-up to #14 suggested by @eriknw . ## Notes for Reviewers I copied these new scripts in `ci/` from the latest state of `branch-24.12` in https://github.com/rapidsai/cugraph, then tried to adapt them to be `nx-cugraph` specific. ### What about notebook tests? Proposing we add notebook tests in a follow-up PR, to keep things smaller and easier to review. Authors: - James Lamb (https://github.com/jameslamb) - Erik Welch (https://github.com/eriknw) Approvers: - Ralph Liu (https://github.com/nv-rliu) - Erik Welch (https://github.com/eriknw) - Kyle Edwards (https://github.com/KyleFromNVIDIA) URL: #15
- Loading branch information
Showing
27 changed files
with
309 additions
and
65 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -15,7 +15,9 @@ jobs: | |
| - changed-files | ||
| - checks | ||
| - conda-python-build | ||
| - conda-python-tests | ||
| - wheel-build-nx-cugraph | ||
| - wheel-tests-nx-cugraph | ||
| secrets: inherit | ||
| uses: rapidsai/shared-workflows/.github/workflows/[email protected] | ||
| if: always() | ||
|
|
@@ -46,14 +48,30 @@ jobs: | |
| uses: rapidsai/shared-workflows/.github/workflows/[email protected] | ||
| with: | ||
| build_type: pull-request | ||
| # This selects "ARCH=amd64 + the latest supported Python + CUDA". | ||
| matrix_filter: map(select(.ARCH == "amd64")) | group_by(.CUDA_VER|split(".")|map(tonumber)|.[0]) | map(max_by([(.PY_VER|split(".")|map(tonumber)), (.CUDA_VER|split(".")|map(tonumber))])) | ||
| conda-python-tests: | ||
| needs: [conda-python-build, changed-files] | ||
| secrets: inherit | ||
| uses: rapidsai/shared-workflows/.github/workflows/[email protected] | ||
| if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python | ||
| with: | ||
| build_type: pull-request | ||
| run_codecov: false | ||
| wheel-build-nx-cugraph: | ||
| needs: [checks] | ||
| secrets: inherit | ||
| uses: rapidsai/shared-workflows/.github/workflows/[email protected] | ||
| with: | ||
| build_type: pull-request | ||
| script: ci/build_wheel_nx-cugraph.sh | ||
| # This selects "ARCH=amd64 + the latest supported Python + CUDA". | ||
| # This selects "ARCH=amd64 + the latest supported Python, 1 job per major CUDA version". | ||
| matrix_filter: map(select(.ARCH == "amd64")) | group_by(.CUDA_VER|split(".")|map(tonumber)|.[0]) | map(max_by([(.PY_VER|split(".")|map(tonumber)), (.CUDA_VER|split(".")|map(tonumber))])) | ||
| wheel-tests-nx-cugraph: | ||
| needs: [wheel-build-nx-cugraph, changed-files] | ||
| secrets: inherit | ||
| uses: rapidsai/shared-workflows/.github/workflows/[email protected] | ||
| if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_python | ||
| with: | ||
| build_type: pull-request | ||
| script: ci/test_wheel_nx-cugraph.sh | ||
| # This selects "ARCH=amd64 + the latest supported Python, 1 job per major CUDA version". | ||
| matrix_filter: map(select(.ARCH == "amd64")) | group_by(.CUDA_VER|split(".")|map(tonumber)|.[0]) | map(max_by([(.PY_VER|split(".")|map(tonumber)), (.CUDA_VER|split(".")|map(tonumber))])) | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -22,6 +22,9 @@ jobs: | |
| branch: ${{ inputs.branch }} | ||
| date: ${{ inputs.date }} | ||
| sha: ${{ inputs.sha }} | ||
| # This selects "ARCH=amd64 + the latest supported Python, 1 job per major CUDA version". | ||
| matrix_filter: map(select(.ARCH == "amd64")) | group_by(.CUDA_VER|split(".")|map(tonumber)|.[0]) | map(max_by([(.PY_VER|split(".")|map(tonumber)), (.CUDA_VER|split(".")|map(tonumber))])) | ||
| run_codecov: false | ||
| wheel-tests-nx-cugraph: | ||
| secrets: inherit | ||
| uses: rapidsai/shared-workflows/.github/workflows/[email protected] | ||
|
|
@@ -31,3 +34,5 @@ jobs: | |
| date: ${{ inputs.date }} | ||
| sha: ${{ inputs.sha }} | ||
| script: ci/test_wheel_nx-cugraph.sh | ||
| # This selects "ARCH=amd64 + the latest supported Python, 1 job per major CUDA version". | ||
| matrix_filter: map(select(.ARCH == "amd64")) | group_by(.CUDA_VER|split(".")|map(tonumber)|.[0]) | map(max_by([(.PY_VER|split(".")|map(tonumber)), (.CUDA_VER|split(".")|map(tonumber))])) | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| #!/bin/bash | ||
| # Copyright (c) 2024, NVIDIA CORPORATION. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| # Support invoking run_nx_cugraph_pytests.sh outside the script directory | ||
| cd "$(dirname "$(realpath "${BASH_SOURCE[0]}")")"/.. | ||
|
|
||
| # Only be verbose and display print information for the first pytest command here | ||
| NX_CUGRAPH_USE_COMPAT_GRAPHS=False pytest --capture=no --verbose --cache-clear --benchmark-disable "$@" ./nx_cugraph/tests | ||
| NX_CUGRAPH_USE_COMPAT_GRAPHS=True pytest --cache-clear --benchmark-disable "$@" ./nx_cugraph/tests |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| #!/bin/bash | ||
| # Copyright (c) 2022-2024, NVIDIA CORPORATION. | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| # Support invoking test_python.sh outside the script directory | ||
| cd "$(dirname "$(realpath "${BASH_SOURCE[0]}")")"/../ | ||
|
|
||
| . /opt/conda/etc/profile.d/conda.sh | ||
|
|
||
| RAPIDS_VERSION="$(rapids-version)" | ||
|
|
||
| rapids-logger "Generate Python testing dependencies" | ||
| rapids-dependency-file-generator \ | ||
| --output conda \ | ||
| --file-key test_python \ | ||
| --matrix "cuda=${RAPIDS_CUDA_VERSION%.*};arch=$(arch);py=${RAPIDS_PY_VERSION}" | tee env.yaml | ||
|
|
||
| rapids-mamba-retry env create --yes -f env.yaml -n test | ||
|
|
||
| # Temporarily allow unbound variables for conda activation. | ||
| set +u | ||
| conda activate test | ||
| set -u | ||
|
|
||
| rapids-logger "Downloading artifacts from previous jobs" | ||
| PYTHON_CHANNEL=$(rapids-download-conda-from-s3 python) | ||
|
|
||
| RAPIDS_TESTS_DIR=${RAPIDS_TESTS_DIR:-"${PWD}/test-results"} | ||
| RAPIDS_COVERAGE_DIR=${RAPIDS_COVERAGE_DIR:-"${PWD}/coverage-results"} | ||
| mkdir -p "${RAPIDS_TESTS_DIR}" "${RAPIDS_COVERAGE_DIR}" | ||
|
|
||
| rapids-print-env | ||
|
|
||
| # TODO: remove the '>=24.12.00a1000' once we start publishing nightly packages | ||
| # from the 'nx-cugraph' repo and stop publishing them from the 'cugraph' repo | ||
| rapids-mamba-retry install \ | ||
| --channel "${PYTHON_CHANNEL}" \ | ||
| "nx-cugraph=${RAPIDS_VERSION}.*,>=24.12.00a1000" | ||
|
|
||
| rapids-logger "Check GPU usage" | ||
| nvidia-smi | ||
|
|
||
| # export LD_PRELOAD="${CONDA_PREFIX}/lib/libgomp.so.1" | ||
|
|
||
| # RAPIDS_DATASET_ROOT_DIR is used by test scripts | ||
| # export RAPIDS_DATASET_ROOT_DIR="$(realpath datasets)" | ||
| # pushd "${RAPIDS_DATASET_ROOT_DIR}" | ||
| # ./get_test_data.sh --benchmark | ||
| # popd | ||
|
|
||
| EXITCODE=0 | ||
| trap "EXITCODE=1" ERR | ||
| set +e | ||
|
|
||
| rapids-logger "pytest nx-cugraph" | ||
| ./ci/run_nx_cugraph_pytests.sh \ | ||
| --junitxml="${RAPIDS_TESTS_DIR}/junit-nx-cugraph.xml" \ | ||
| --cov=nx_cugraph \ | ||
| --cov-report=xml:"${RAPIDS_COVERAGE_DIR}/nx-cugraph-coverage.xml" \ | ||
| --cov-report=term | ||
|
|
||
| rapids-logger "pytest networkx using nx-cugraph backend" | ||
|
|
||
| pushd nx_cugraph | ||
| ../run_nx_tests.sh | ||
|
|
||
| # run_nx_tests.sh outputs coverage data, so check that total coverage is >0.0% | ||
| # in case nx-cugraph failed to load but fallback mode allowed the run to pass. | ||
| _coverage=$(coverage report | grep "^TOTAL") | ||
|
|
||
| echo "nx-cugraph coverage from networkx tests: $_coverage" | ||
| echo $_coverage | awk '{ if ($NF == "0.0%") exit 1 }' | ||
|
|
||
| # Ensure all algorithms were called by comparing covered lines to function lines. | ||
| # Run our tests again (they're fast enough) to add their coverage, then create coverage.json | ||
| NX_CUGRAPH_USE_COMPAT_GRAPHS=False pytest \ | ||
| --pyargs nx_cugraph \ | ||
| --config-file=../pyproject.toml \ | ||
| --cov-config=../pyproject.toml \ | ||
| --cov=nx_cugraph \ | ||
| --cov-append \ | ||
| --cov-report= | ||
|
|
||
| coverage report \ | ||
| --include="*/nx_cugraph/algorithms/*" \ | ||
| --omit=__init__.py \ | ||
| --show-missing \ | ||
| --rcfile=../pyproject.toml | ||
|
|
||
| coverage json --rcfile=../pyproject.toml | ||
|
|
||
| python -m nx_cugraph.tests.ensure_algos_covered | ||
|
|
||
| # Exercise (and show results of) scripts that show implemented networkx algorithms | ||
| python -m nx_cugraph.scripts.print_tree --dispatch-name --plc --incomplete --different | ||
| python -m nx_cugraph.scripts.print_table | ||
| popd | ||
|
|
||
| rapids-logger "Test script exiting with value: $EXITCODE" | ||
| exit ${EXITCODE} |
Oops, something went wrong.