-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix azimuth, split same-date ARs, new output CSVs, finish QC notebook #18
Conversation
Hey @charparr and @kyleredilla - I pushed a lot more changes to this I think the merge conflict here is entirely due to the fact that I deleted one of the notebooks. 🤞 If we need to work through that together, please let me know! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I successfully executed the pipeline here! I performed the following steps:
- Resolved merge conflicts
- Created a fresh environment using the
mamba
package manager. I asked for the following packages:
xarray
pyproj
geopandas
pandas
tqdm
scipy
skimage
haversine
shapely
rasterio
cdsapi
matplotlib
seaborn
jupyterlab
dask
scikit-image
rioxarray
I did this because creating the environment from the existing .yml
file was taking forever - this is a known conda
quirk and is the raison d'être for alternative package managers like mamba
.
- Executed the
download.py
script - while not strictly needed for this PR, I did this because I was working with a different env - Executed the
compute_ivt.py
script without providing arguments (i.e. using the defaults). This computed 90th percentile IVT values. - Executed the
ar_detection.py
script without providing arguments (i.e. using the defaults). - Executed the
AR_QC.ipynb
notebook - Executed the
AR_avalanche_exploration.ipynb
notebook
All of these scripts and notebooks executed successfully with zero issues!
This is a great body of work and as far as I can tell it is ready for a "production" run, meaning running the pipeline from 1980 onward and then sharing results with our collaborators.
The only change I'll request here is to settle the python environment to converge on a more easily resolvable package spec.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We'll add that list of packages to the README and in the subsequent branch do the following to settle the env:
- Create a new conda env named “atmospheric_rivers” or similar
- Test in a production run and in review
- Replace environment.yml file accordingly
This (resubmitted) PR:
environment.yml
fileThis PR closes #16 , closes #17 , and closes #10 . I couldn't find a better CRS to use than EPSG:3338, and since the spatial analysis with avalanche events will focus on Alaska, I think we just go with that (closes #7 ).
This PR makes no changes to
download.py
. So if you have already downloaded the data, you can skip that step. However, thecompute_ivt.py
should be run again in order to use the 90th percentile IVT.Instructions: The
README
andconfig.py
files should provide the basic setup for testing this PR. To test:export AR_DATA_DIR=...
environment.yml
and activate itshp
andcsv
outputs from previous model runsconfig.py
file, making sure to use90
forivt_percentile
python download.py
(optional)python compute_ivt.py
AR_QC.ipynb
notebookNote that Nathan requested data from 1980 to present, to match up with the date range of his avalanche database. In the interest of time, I'd like you to just test this PR using the data already downloaded; I will run the whole pipeline from 1980-present and package that data for Nathan.