Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cache NUMBA kernels between CI runs #279

Merged
merged 50 commits into from
Jan 31, 2024
Merged
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
eb8c568
Cache NUMBA kernels between CI runs
sjperkins Jun 15, 2023
dd6bbeb
Use actions/cache@v3
sjperkins Jun 15, 2023
d4cba34
Cache per python version
sjperkins Jun 15, 2023
3ab69ad
runner.tmp -> runner.temp
sjperkins Jun 15, 2023
353ae63
Debugging
sjperkins Jun 15, 2023
a60360d
Fix
sjperkins Jun 15, 2023
707b176
Run entire test suite
sjperkins Jun 15, 2023
5c3aed0
timestamp needed otherwise cache hit occurs and cache not updated
sjperkins Jun 15, 2023
edc58e8
Fix output
sjperkins Jun 15, 2023
7ef4776
Add revert_me.txt
sjperkins Jun 15, 2023
55fd7ba
Merge branch 'main' into cache-numba-kernels
JSKenyon Jun 20, 2023
d9f69ec
Use nearest-neighbour interpolation in regions where extrapolation is…
JSKenyon Jul 26, 2023
0c58b33
Utilise environment variable when dask.address is unset. (#288)
JSKenyon Aug 3, 2023
6a9fcf4
Add plotting functionality (#290)
JSKenyon Aug 25, 2023
7cd19eb
Fix #293 - OOB access caused by `output.subtract_directions` (#294)
JSKenyon Aug 30, 2023
d0b6653
Namedbackups (#296)
landmanbester Sep 13, 2023
6bed6f2
Selectively disable MAD flagging criteria (#298)
JSKenyon Oct 3, 2023
7630360
Disable mad flagging on off-diagonals by default (#300)
JSKenyon Oct 19, 2023
eaa7515
Fix bug affecting non-standard columns in `input_ms.data_column` (#301)
JSKenyon Oct 20, 2023
81a2ef8
Don't allow restore app to overwrite metadata (#307)
landmanbester Oct 25, 2023
5478340
Fix for summary reporting SOURCE_ID as FIELD_ID (#309)
JSKenyon Nov 9, 2023
9a3be80
Fix receptor summary (#310)
JSKenyon Nov 13, 2023
84e317a
Fix xarray dims (#318)
JSKenyon Jan 26, 2024
e7e03f1
Fixes for changes relating to Numba error types. (#319)
JSKenyon Jan 26, 2024
abb60db
Move now-deprecated graph metrics function into the scheduler plugin …
JSKenyon Jan 26, 2024
20939b3
Make small changes to enable 3.11 compatibilty. Requires changes in s…
JSKenyon Jan 29, 2024
ec4cf8b
Restringify keys in scheduler plugin. (#322)
JSKenyon Jan 29, 2024
8008ab3
Merge branch 'v0.2.1-dev' into cache-numba-kernels
JSKenyon Jan 29, 2024
2522829
Attempt very dodgy solution to caching problem.
JSKenyon Jan 29, 2024
6e13b25
Look for code in the correct place.
JSKenyon Jan 29, 2024
3eb7c5d
Update pyproject.toml. Add poetry.lock. Update docs. (#323)
JSKenyon Jan 29, 2024
79158e2
Some debugging.
JSKenyon Jan 29, 2024
92da770
Merge v0.2.1-dev.
JSKenyon Jan 29, 2024
e83fd40
Fix unsaved file.
JSKenyon Jan 29, 2024
6348e00
More debugging.
JSKenyon Jan 29, 2024
92326a7
Temporarily make test suite much smaller.
JSKenyon Jan 29, 2024
d1c67a9
Fix path.
JSKenyon Jan 29, 2024
16bdee5
Actually fix path.
JSKenyon Jan 29, 2024
c2a3b29
Attempt at safer caching.
JSKenyon Jan 31, 2024
0e97a1c
Merge in v0.2.2-dev
JSKenyon Jan 31, 2024
32f5950
More fiddling with paths.
JSKenyon Jan 31, 2024
3db9902
Fix bad tabbing.
JSKenyon Jan 31, 2024
61f61b5
Try to find out where things are failing.
JSKenyon Jan 31, 2024
4d1dca0
More fiddling.
JSKenyon Jan 31, 2024
3d21087
More fiddling.
JSKenyon Jan 31, 2024
6318f47
More fiddling.
JSKenyon Jan 31, 2024
f37fa83
Try restore time action.
JSKenyon Jan 31, 2024
d270cff
Tidy up caching approach. Use action. Restore matrix and test everyth…
JSKenyon Jan 31, 2024
1da4879
Remove tmp file.
JSKenyon Jan 31, 2024
65c5efa
Reword CI step name.
JSKenyon Jan 31, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 24 additions & 1 deletion .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@ jobs:
runs-on: ${{ matrix.os }}
continue-on-error: true

env:
NUMBA_CACHE_DIR: /tmp/numba-cache

strategy:
matrix:
os: [ubuntu-20.04, ubuntu-22.04]
Expand All @@ -38,7 +41,27 @@ jobs:
- name: Checkout source
uses: actions/checkout@v3
with:
fetch-depth: 1
fetch-depth: 0 # Full fetch required to restore times.

- name: Restore repo times
uses: chetan/git-restore-mtime-action@v2

# Upgrade pip and setuptools
- name: Upgrade pip and setuptools
run: python -m pip install -U pip setuptools

- name: Create Key and Numba Cache Directory
id: numba-key
run: |
mkdir -p ${{ env.NUMBA_CACHE_DIR }}
echo "timestamp=$(/bin/date -u '+%Y%m%d%H%M%S')" >> $GITHUB_OUTPUT

- name: Cache Numba Kernels
uses: actions/cache@v3
with:
key: numba-cache-${{ matrix.python-version }}-${{ steps.numba-key.outputs.timestamp }}
restore-keys: numba-cache-${{ matrix.python-version }}-
path: ${{ env.NUMBA_CACHE_DIR }}

- name: List the measures directory
run: curl ftp://ftp.astron.nl/outgoing/Measures/ > measures_dir.txt
Expand Down
Loading