Base.Filesystem.rm
Base.convertSpaceLiDAR.angle!SpaceLiDAR.boundsSpaceLiDAR.boundsSpaceLiDAR.boundsSpaceLiDAR.classifySpaceLiDAR.downloadSpaceLiDAR.downloadSpaceLiDAR.download!SpaceLiDAR.download!SpaceLiDAR.infoSpaceLiDAR.infoSpaceLiDAR.infoSpaceLiDAR.infoSpaceLiDAR.instantiateSpaceLiDAR.isvalidSpaceLiDAR.netrc!SpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.searchSpaceLiDAR.shiftSpaceLiDAR.syncpoints(g::ICESat_Granule{:GLAH14}, step=1, bbox::Union{Nothing,Extent,NamedTuple} = nothing)
@@ -1133,7 +1133,7 @@ ICESatICESat-2¶
@@ -1142,7 +1142,7 @@ ICESat-2classify(granule::ICESat2_Granule{:ATL03}, atl08::Union{ICESat2_Granule{:ATL08},Nothing} = nothing, tracks = icesat2_tracks)
Like points(::ICESat2_Granule{:ATL03}) but with the classification from the ATL08 dataset. If an ATL08 granule is not provided, we try to find it based on the ATL03 name using convert.
-
+
#
SpaceLiDAR.points — Method.
points(g::ICESat2_Granule{:ATL03}, tracks=icesat2_tracks; step=1, bbox::Union{Nothing,Extent,NamedTuple} = nothing)
@@ -1239,7 +1239,7 @@ ICESat-2points(g::ICESat2_Granule{:ATL06}, tracks=icesat2_tracks, step=1, bbox::Union{Nothing,Extent,NamedTuple} = nothing)
@@ -1324,7 +1324,7 @@ ICESat-2points(g::ICESat2_Granule{:ATL08}; tracks=icesat2_tracks, step=1, canopy=false, ground=true, bbox::Union{Nothing,Extent,NamedTuple} = nothing)
@@ -1440,7 +1440,7 @@ ICESat-2points(g::ICESat2_Granule{:ATL12}, tracks=icesat2_tracks)
@@ -1501,7 +1501,7 @@ ICESat-2GEDI¶
@@ -1514,7 +1514,7 @@ GEDI&par
Warning
This opens the .h5 file to read all tracks, so it is very slow.
-
+
diff --git a/dev/reference/granules/index.html b/dev/reference/granules/index.html
index 55f74fe..5142e87 100644
--- a/dev/reference/granules/index.html
+++ b/dev/reference/granules/index.html
@@ -927,33 +927,33 @@ Granulesrm(granule::Granule)
Remove the file associated with granule from the local filesystem.
-
+
#
SpaceLiDAR.download — Function.
Like download, but for a vector of granules.
-
+
#
SpaceLiDAR.download — Function.
Download the file associated with granule to the folder, from an http(s) location if it doesn't already exists locally. Returns a new granule. See download! for a mutating version.
Will require credentials (netrc) which can be set with netrc!.
-
+
#
SpaceLiDAR.download! — Function.
-download!(granules::Vector{<:Granule}, folder=".")
+
-Like download!, but for a vector of granules. Will make use of aria2c (parallel).
-
+Download the file associated with granule to the folder, from an http(s) location if it doesn't already exists locally.
+Will require credentials (netrc) which can be set with netrc!.
+
#
SpaceLiDAR.download! — Function.
-download!(granule::Granule, folder=".")
+
-Download the file associated with granule to the folder, from an http(s) location if it doesn't already exists locally.
-Will require credentials (netrc) which can be set with netrc!.
-
+Like download!, but for a vector of granules. Will make use of aria2c (parallel).
+
#
SpaceLiDAR.sync — Function.
sync(folder::AbstractString, all::Bool=false; kwargs...)
@@ -969,31 +969,31 @@ Granulessync(::Symbol, folders, all) instead.
When all is false (the default), sync will search only for granules past the date of the latest granule found in folders. If true, it will search for all granules. Note that ICESat granules are not timestamped, so sync will try to download all ICESat granules not yet present, regardless of this setting.
Any kwargs... are passed to the search function. This enables sync to only download granules within a certain extent, for example.
-
+
#
SpaceLiDAR.ICESat_Granule — Type.
A granule of the ICESat product product. Normally created automatically from either find, granule_from_file or granules_from_folder.
-
+
#
SpaceLiDAR.info — Method.
Derive info based on the filename. The name is built up as follows: ATL03[yyyymmdd][hhmmss][ttttccss][vvvrr].h5. See section 1.2.5 in the user guide.
-
+
#
SpaceLiDAR.ICESat2_Granule — Type.
A granule of the ICESat-2 product product. Normally created automatically from either find, granule_from_file or granules_from_folder.
-
+
#
Base.convert — Method.
Converts the granule g to the product product, by guessing the correct name.
-
+
#
SpaceLiDAR.bounds — Method.
bounds(granule::ICESat2_Granule)
@@ -1007,25 +1007,25 @@ Granulesjulia> bounds(g)
g = ICESat2_Granule()
-
+
#
SpaceLiDAR.info — Method.
Derive info based on the filename. The name is built up as follows: ATL03_[yyyymmdd][hhmmss]_[ttttccss]_[vvv_rr].h5. See section 1.2.5 in the user guide.
-
+
#
SpaceLiDAR.GEDI_Granule — Type.
A granule of the GEDI product product. Normally created automatically from either find, granule_from_file or granules_from_folder.
-
+
#
SpaceLiDAR.info — Method.
Derive info based on the filename. This is built up as follows: GEDI02_A_2019110014613_O01991_T04905_02_001_01.h5 or in case of v"2": GEDI02_A_2019242104318_O04046_01_T02343_02_003_02_V002.h5. See section 2.4 in the user guide.
-
+
diff --git a/dev/reference/search/index.html b/dev/reference/search/index.html
index b4b296d..2f9f771 100644
--- a/dev/reference/search/index.html
+++ b/dev/reference/search/index.html
@@ -928,7 +928,7 @@ Searchsearch(:GEDI02_A, "002") # searches *all* GEDI v2 granules
Search granules for a given mission and bounding box.
-
+
diff --git a/dev/reference/utilities/index.html b/dev/reference/utilities/index.html
index 0469a86..510a1ec 100644
--- a/dev/reference/utilities/index.html
+++ b/dev/reference/utilities/index.html
@@ -927,37 +927,37 @@ Utilitiesangle!(table)
Sets the angle column in table as returned from points. See angle for details.
-
+
#
SpaceLiDAR.shift — Method.
Shift longitude and latitude with distance in [m] in direction angle, where North is 0°. Returns a tuple of the shifted coordinates: (longitude, latitude). Useful for offsetting SpaceLiDAR points to the left or right of the track, in combination with angle.
-
+
#
SpaceLiDAR.to_egm2008! — Method.
Converts ellipsoid heights to geoid heights using the EGM2008 geoid model. Assumes a table as generated from points with columns :latitude, :longitude, and :height. Will overwrite the :height column with the geoid height.
-
+
<a id='SpaceLiDAR.instantiate-Union{Tuple{T}, Tuple{Vector{T}, AbstractString}} where T<:SpaceLiDAR.Granule' href='#SpaceLiDAR.instantiate-Union{Tuple{T}, Tuple{Vector{T}, AbstractString}} where T<:SpaceLiDAR.Granule'>#
SpaceLiDAR.instantiate — Method.
For a given list of granules from find, match the granules to the local files and return a new list of granules with the local filepaths if they exist.
-
+
#
SpaceLiDAR.isvalid — Method.
Checks if a granule is has a valid, local and non-corrupt .h5 file. Can be combined with rm(::Granule) to remove invalid granules.
-
+
#
SpaceLiDAR.netrc! — Method.
Writes/updates a .netrc file for ICESat-2 and GEDI downloads. A .netrc is a plaintext file containing your username and password for NASA EarthData and DAACs, and can be automatically used by Julia using Downloads and tools like wget, curl among others.
-
+
diff --git a/dev/search/search_index.json b/dev/search/search_index.json
index 1efca8c..dbe9191 100644
--- a/dev/search/search_index.json
+++ b/dev/search/search_index.json
@@ -1 +1 @@
-{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":""},{"location":"#spacelidar","title":"SpaceLiDAR","text":"A Julia toolbox for ICESat, ICESat-2 and GEDI data. Quickly search, download and load filtered point data with relevant attributes from the .h5 granules of each data product. For an overview with code examples, see the FOSS4G Pluto notebook here.
If you use SpaceLiDAR in your research, please consider citing it.
"},{"location":"#supported-data-products","title":"Supported data products","text":"Currently supports the following data products:
mission data product User Guide (UG) Algorithm Theoretical Basis Document (ATBD) ICESat GLAH06 v34 UG ATBD ICESat GLAH14 v34 UG ATBD ICESat-2 ATL03 v6 UG ATBD ICESat-2 ATL06 v5 UG ATBD ICESat-2 ATL08 v6 UG ATBD ICESat-2 ATL12 v5 UG ATBD GEDI L2A v2 UG ATBD "},{"location":"#documentation","title":"Documentation","text":"This documentation is set up using the Divio documentation system and thus split into Tutorials, Guides, Topics and References.
"},{"location":"#publications","title":"Publications","text":"The code produced for the following paper was the beginning of this package:
Vernimmen, Ronald, Aljosja Hooijer, and Maarten Pronk. 2020. \u2018New ICESat-2 Satellite LiDAR Data Allow First Global Lowland DTM Suitable for Accurate Coastal Flood Risk Assessment\u2019. Remote Sensing 12 (17): 2827. https://doi.org/10/gg9dg6.
The DTM produced using ICESat-2 ATL08 data was in turn used for:
Hooijer, A., and R. Vernimmen. 2021. \u2018Global LiDAR Land Elevation Data Reveal Greatest Sea-Level Rise Vulnerability in the Tropics\u2019. Nature Communications 12 (1): 3592. https://doi.org/10/gkzf49.
"},{"location":"changelog/","title":"Changelog","text":""},{"location":"changelog/#unreleased","title":"Unreleased","text":" - Working on generic retrieval of parameters as a Table from a granule, instead of the hardcoded choices made now for each product. Would result in methods like
points(granule, [vara, varb]).
"},{"location":"changelog/#new-features","title":"New features","text":" - New types
Table and PartitionedTable, which support the Tables.jl interface. This prevents allocating code like reduce(vcat, DataFrame.(points(granule))) to get a DataFrame. You can now just call DataFrame(table). - Reduced allocations in retrieving point data.
- Introduced
before and after keywords in search, to search by date(ranges).
"},{"location":"changelog/#fixed","title":"Fixed","text":" - Empty (filtered) granules could result in
Vector{BitVector} columns, which have been changed to Vector{Bool}.
"},{"location":"changelog/#breaking","title":"Breaking","text":" points now return either a Table or a PartitionedTable instead of NamedTuple or Vector{NamedTuple}. The old behaviour can be regained by calling parent on these tables.- Removed
number_of_returns and return_number from ICESat-2 ATL08 canopy output.
"},{"location":"changelog/#deprecated","title":"Deprecated","text":" - Renamed
granule_from_file to granule - Renamed
granules_from_file to granules - Renamed
write_granule_urls! to write_urls
"},{"location":"changelog/#changed","title":"Changed","text":" - Most of the search functionality has been moved out to the more generic EarthData.jl
- Updated ICESat-2 from version 5 to version 6
"},{"location":"changelog/#v030","title":"v0.3.0","text":" - GeoInterface, Extents support
- Bounding box using Extent subsetting on all
points functions find renamed to searchsearch now takes the product as a Symbol and accepts a s3::Bool parameter.- Stricter checking of arguments in
search - MultiPolygon extents of granules are now returned in the
polygons field in granules from search. - Removed
bbox field on granules. download(granule) now works on AWS S3 urlsdonwload(granule) now creates temporary files and renames after successful download- ICESat-2 ATL08 v5
points function now supports 20 m resolution by passing highres::Bool parameter.
"},{"location":"changelog/#v022","title":"v0.2.2","text":" - Fixed ICESat-2 download (please remove
n5eil01u.ecs.nsidc.org from your .netrc file) - Linked to Zenodo for DOI citations
"},{"location":"changelog/#v021","title":"v0.2.1","text":" - Unified
bounds of granules - Fixed
getcoord for Point and added Point(x, y, z) constructor - Updated utils to make use of DataFrames with correct column names
"},{"location":"changelog/#v020","title":"v0.2.0","text":"Warning
This is a breaking release
- Many of the column names have changed to be more descriptive.
- Documentation and docstring improvements.
- Tables support, you can now do
DataFrame(granule), without having to call points(granule). - Memory use improvements, by using SentinelArray of FillArray under the hood.
- Dropped S3, GeoArrays and LAS/LAZ support.
- Added GeoInterface support for lines/points and dropped GeoDataFrames
- Expanded test coverage.
"},{"location":"changelog/#v016","title":"v0.1.6","text":" - Support for ICESat GLAH06 by alex-s-gardner
"},{"location":"changelog/#v015","title":"v0.1.5","text":" - Support for ICESat-2 ATL06
- Update search to use v5 for ICESat-2 by default
"},{"location":"changelog/#v014","title":"v0.1.4","text":" - Compatibility fixes
"},{"location":"changelog/#v013","title":"v0.1.3","text":" - Added interpolation for GeoArrays
- Added FOSS4G notebook
"},{"location":"guides/downloads/","title":"Downloading","text":""},{"location":"guides/downloads/#downloading","title":"Downloading","text":"As shown in Tutorial: Search for data, it is easy to find data. Downloading the data might be harder though, especially when downloading a large amount of granules or even mirroring a complete DAAC. Indeed, the Julia download! won't work in parallel, nor will it resume downloads or show its progress. In such cases it's useful to export a list of granules to a text file and use an external download tool:
granules = find(:ICESat2, \"ATL08\")\nSpaceLiDAR.write_urls(\"atl08_world.txt\", granules)\n
In my case, I use aria2c. Note that downloading from the granule urls require a EarthData login, normally setup in an .netrc file (also see netrc!).
aria2c -c -i atl08_world.txt\n
Once finished, one can again instantiate the list of granules with the folder to which all files have been downloaded.
"},{"location":"guides/tracks/","title":"Track filtering","text":""},{"location":"guides/tracks/#track-filtering","title":"Track filtering","text":"Coming soon
"},{"location":"reference/api/","title":"Index","text":""},{"location":"reference/api/#api-reference","title":"API reference","text":" SpaceLiDAR.GEDI_GranuleSpaceLiDAR.ICESat2_GranuleSpaceLiDAR.ICESat_GranuleBase.Filesystem.rmBase.convertSpaceLiDAR.angle!SpaceLiDAR.boundsSpaceLiDAR.boundsSpaceLiDAR.classifySpaceLiDAR.downloadSpaceLiDAR.downloadSpaceLiDAR.download!SpaceLiDAR.download!SpaceLiDAR.infoSpaceLiDAR.infoSpaceLiDAR.infoSpaceLiDAR.instantiateSpaceLiDAR.isvalidSpaceLiDAR.netrc!SpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.searchSpaceLiDAR.shiftSpaceLiDAR.syncSpaceLiDAR.to_egm2008!
"},{"location":"reference/data/","title":"Data","text":""},{"location":"reference/data/#data","title":"Data","text":""},{"location":"reference/data/#icesat","title":"ICESat","text":"# SpaceLiDAR.points \u2014 Method.
points(g::ICESat_Granule{:GLAH06}, step=1, bbox::Union{Nothing,Extent,NamedTuple} = nothing)\n
Retrieve the points for a given ICESat GLAH06 (Land Ice) granule as a list of namedtuples The names of the tuples are based on the following fields:
Variable Original Field Description Units longitude Data_40HZ/Geolocation/d_lon Longitude of segment center, WGS84, East=+ decimal degrees latitude Data_40HZ/Geolocation/d_lat Latitude of segment center, WGS84, North=+ decimal degrees height Data_40HZ/Elevation_Surfaces/d_elev + Data_40HZ/Elevation_Corrections/d_satElevCorr m above WGS84 ellipsoid datetime Data_40HZ/DS_UTCTime_40 Precise time of aquisiton date-time quality 1 Data_40HZ/Quality/elev_use_flg & Data_40HZ/Quality/sigma_att_flg = 0 & Data_40HZ/Waveform/i_numPk = 1 & Data_40HZ/Elevation_Corrections/d_satElevCorr < 3 1 = high quality height_reference land_ice_segments/dem/dem_h Height of the (best available) DEM height above WGS84 You can get the output in a DataFrame with DataFrame(points(g)).
source
# SpaceLiDAR.points \u2014 Method.
points(g::ICESat_Granule{:GLAH14}, step=1, bbox::Union{Nothing,Extent,NamedTuple} = nothing)\n
Retrieve the points for a given ICESat GLAH14 (Land Surface) granule as a list of namedtuples The names of the tuples are based on the following fields:
Variable Original Field (in Data_40HZ) Description Units longitude /Geolocation/d_lon Longitude of segment center, WGS84, East=+ decimal degrees latitude Geolocation/d_lat Latitude of segment center, WGS84, North=+ decimal degrees height Elevation_Surfaces/d_elev + Elevation_Corrections/d_satElevCorr m above WGS84 ellipsoid datetime DS_UTCTime_40 Precise time of aquisiton date-time quality 1 Quality/elev_use_flg & Quality/sigma_att_flg = 0 & Waveform/i_numPk = 1 & Elevation_Corrections/d_satElevCorr < 3 1=high quality clouds Elevation_Flags/elv_cloud_flg Cloud contamination - height_reference Geophysical/d_DEM_elv Height of the (best available) DEM height above WGS84 gain Waveform/i_gval_rcv Gain value used for received pulse. - reflectivity Reflectivity/d_reflctUC Reflectivity, not corrected - attitude Quality/sigma_att_flg Attitude quality indicator 0=good; 50=warning; 100=bad; saturation Quality/sat_corr_flg Saturation Correction Flag 0=not_saturated; You can get the output in a DataFrame with DataFrame(points(g)).
source
"},{"location":"reference/data/#icesat-2","title":"ICESat-2","text":"# SpaceLiDAR.classify \u2014 Function.
classify(granule::ICESat2_Granule{:ATL03}, atl08::Union{ICESat2_Granule{:ATL08},Nothing} = nothing, tracks = icesat2_tracks)\n
Like points(::ICESat2_Granule{:ATL03}) but with the classification from the ATL08 dataset. If an ATL08 granule is not provided, we try to find it based on the ATL03 name using convert.
source
# SpaceLiDAR.points \u2014 Method.
points(g::ICESat2_Granule{:ATL03}, tracks=icesat2_tracks; step=1, bbox::Union{Nothing,Extent,NamedTuple} = nothing)\n
Retrieve the points for a given ICESat-2 ATL03 (Global Geolocated Photon Data) granule as a list of namedtuples, one for each beam. The names of the tuples are based on the following fields:
Column Field Description Units longitude heights/lon_ph Longitude of photon, WGS84, East=+ decimal degrees latitude heights/lat_ph Latitude of photon, WGS84, North=+ decimal degrees height heights/h_ph Photon WGS84 Height m above the WGS 84 ellipsoid quality heights/quality_ph Indicates the quality of the associated photon 0 = nominal uncertainty geolocation/sigma_h Estimated height uncertainty m datetime heights/delta_time + ancillary_data/atlas_sdp_gps_epoch + gps_offset date-time confidence heights/signal_conf_ph Photon Signal Confidence 2=low; 3=med; 4=high segment geolocation/segment_id Along-track segment ID number - track gt1l - gt3r groups - - strong_beam - \"strong\" (true) or \"weak\" (false) laser power - sun_angle geolocation/solar_elevation Sun angle \u00b0 above horizon detector_id atlas_spot_number attribute - - height_reference heights/dem/dem_h Height of the (best available) DEM m above the WGS 84 ellipsoid You can combine the output in a DataFrame with reduce(vcat, DataFrame.(points(g))) if you want to change the default arguments or DataFrame(g) with the default options.
source
# SpaceLiDAR.points \u2014 Method.
points(g::ICESat2_Granule{:ATL06}, tracks=icesat2_tracks, step=1, bbox::Union{Nothing,Extent,NamedTuple} = nothing)\n
Retrieve the points for a given ICESat-2 ATL06 (Land Ice) granule as a list of namedtuples, one for each beam. The names of the tuples are based on the following fields:
Column Field Description Units longitude land_ice_segments/longitude Longitude of segment center, WGS84, East=+ decimal degrees latitude land_ice_segments/latitude Latitude of segment center, WGS84, North=+ decimal degrees height land_ice_segments/h_li Standard land-ice segment height m above the WGS 84 ellipsoid height_error \u221a(land_ice_segments/sigma_geo_h\u00b2 + Total vertical geolocation error m above the WGS 84 ellipsoid land_ice_segments/h_li_sigma\u00b2) datetime land_ice_segments/delta_time + ancillary_data/atlas_sdp_gps_epoch + gps_offset date-time quality land_ice_segments/atl06_quality_summary Boolean flag indicating the best-quality subset 1 = high quality track gt1l - gt3r groups - - strong_beam - \"strong\" (true) or \"weak\" (false) laser power - detector_id atlas_spot_number attribute - - height_reference land_ice_segments/dem/dem_h Height of the (best available) DEM - You can combine the output in a DataFrame with reduce(vcat, DataFrame.(points(g))) if you want to change the default arguments or DataFrame(g) with the default options.
source
# SpaceLiDAR.points \u2014 Method.
points(g::ICESat2_Granule{:ATL08}; tracks=icesat2_tracks, step=1, canopy=false, ground=true, bbox::Union{Nothing,Extent,NamedTuple} = nothing)\n
Retrieve the points for a given ICESat-2 ATL08 (Land and Vegetation Height) granule as a list of namedtuples, one for each beam. With the tracks keyword, you can specify which tracks to include. The default is to include all tracks. With the step keyword, you can choose to limit the number of points, the default is 1 (all points). With setting ground and or canopy, you can control to include ground and/or canopy points. Finally, with the ground_field and canopy_field settings, you can determine the source field. The default is h_te_mean for ground and h_mean_canopy_abs for canopy. With the introduction of v5, a 20m resolution is also available for estimation, which you can enable with highres. Note that filtering with a bounding box doesn't yet work when highres is true.
The names of the tuples are based on the following fields:
Column Field Description Units longitude land_segments/longitude Longitude of segment center, WGS84, East=+ decimal degrees latitude land_segments/latitude Latitude of segment center, WGS84, North=+ decimal degrees height land_segments/terrain/h_te_mean Standard land-ice segment height m above the WGS 84 ellipsoid height_error land_segments/terrain/h_te_uncertainty Total vertical geolocation error m datetime land_segments/delta_time + ancillary_data/atlas_sdp_gps_epoch + gps_offset date-time quality land_segments/terrain_flg Boolean flag indicating the best-quality subset 1 = high quality phr land_segments/ph_removal_flag More than 50% of photons removed - sensitivity land_segments/snr The signal to noise ratio - scattered land_segments/msw_flag Multiple Scattering warning flag -1=unknown; 0=none saturated land_segments/sat_flag Saturation detected - clouds land_segments/layer_flag Clouds or blowing snow are likely present - track gt1l - gt3r groups - - strong_beam - \"strong\" (true) or \"weak\" (false) laser power - classification - \"ground\", \"high_canopy\" - height_reference land_segments/dem_h Height of the (best available) DEM m above the WGS 84 ellipsoid detector_id atlas_spot_number attribute - - You can combine the output in a DataFrame with reduce(vcat, DataFrame.(points(g))) if you want to change the default arguments or just DataFrame(g) with the default options.
source
# SpaceLiDAR.points \u2014 Function.
points(g::ICESat2_Granule{:ATL12}, tracks=icesat2_tracks)\n
Retrieve the points for a given ICESat-2 ATL12 (Ocean Surface Height) granule as a list of namedtuples, one for each beam. The names of the tuples are based on the following fields:
Column Field Description Units longitude ssh_segments/longitude Longitude of segment center, WGS84, East=+ decimal degrees latitude ssh_segments/latitude Latitude of segment center, WGS84, North=+ decimal degrees height ssh_segments/heights/h Standard land-ice segment height m above the WGS 84 ellipsoid datetime ssh_segments/delta_time + ancillary_data/atlas_sdp_gps_epoch + gps_offset date-time track gt1l - gt3r groups - - strong_beam - \"strong\" (true) or \"weak\" (false) laser power - detector_id atlas_spot_number attribute - - You can combine the output in a DataFrame with reduce(vcat, DataFrame.(points(g))) if you want to change the default arguments or DataFrame(g) with the default options.
source
"},{"location":"reference/data/#gedi","title":"GEDI","text":"# SpaceLiDAR.bounds \u2014 Method.
bounds(granule::GEDI_Granule)\n
Return the bounds of the GEDI granule.
Warning
This opens the .h5 file to read all tracks, so it is very slow.
source
-
Smith, B., Fricker, H. A., Gardner, A. S., Medley, B., Nilsson, J., Paolo, F. S., ... & Zwally, H. J. (2020). Pervasive ice sheet mass loss reflects competing ocean and atmosphere processes. Science, 368(6496), 1239-1242.\u00a0\u21a9\u21a9
"},{"location":"reference/granules/","title":"Granules","text":""},{"location":"reference/granules/#granules","title":"Granules","text":"# Base.Filesystem.rm \u2014 Method.
rm(granule::Granule)\n
Remove the file associated with granule from the local filesystem.
source
# SpaceLiDAR.download \u2014 Function.
download(granules::Vector{<:Granule}, folder=\".\")\n
Like download, but for a vector of granules.
source
# SpaceLiDAR.download \u2014 Function.
download(granule::Granule, folder=\".\")\n
Download the file associated with granule to the folder, from an http(s) location if it doesn't already exists locally. Returns a new granule. See download! for a mutating version.
Will require credentials (netrc) which can be set with netrc!.
source
# SpaceLiDAR.download! \u2014 Function.
download!(granules::Vector{<:Granule}, folder=\".\")\n
Like download!, but for a vector of granules. Will make use of aria2c (parallel).
source
# SpaceLiDAR.download! \u2014 Function.
download!(granule::Granule, folder=\".\")\n
Download the file associated with granule to the folder, from an http(s) location if it doesn't already exists locally.
Will require credentials (netrc) which can be set with netrc!.
source
# SpaceLiDAR.sync \u2014 Function.
sync(folder::AbstractString, all::Bool=false; kwargs...)\nsync(folders::AbstractVector{<:AbstractString}, all::Bool=false; kwargs...)\nsync(product::Symbol, folder::AbstractString, all::Bool=false; kwargs...)\nsync(product::Symbol, folders::AbstractVector{<:AbstractString}, all::Bool=false; kwargs...)\n
Syncronize an existing archive of local granules in folder(s) with the latest granules available. Specifically, this will run search and download for any granules not yet present in folder(s), to the first folder in the list.
Warning
Using sync could result in downloading significant (TB+) amounts of data.
Assumes all folders contain granules of the same product. If not, pass the product as Symbol: sync(::Symbol, folders, all) instead.
When all is false (the default), sync will search only for granules past the date of the latest granule found in folders. If true, it will search for all granules. Note that ICESat granules are not timestamped, so sync will try to download all ICESat granules not yet present, regardless of this setting.
Any kwargs... are passed to the search function. This enables sync to only download granules within a certain extent, for example.
source
# SpaceLiDAR.ICESat_Granule \u2014 Type.
ICESat_Granule{product} <: Granule\n
A granule of the ICESat product product. Normally created automatically from either find, granule_from_file or granules_from_folder.
source
# SpaceLiDAR.info \u2014 Method.
info(g::ICESat_Granule)\n
Derive info based on the filename. The name is built up as follows: ATL03yyyymmdd[ttttccss][vvvrr].h5. See section 1.2.5 in the user guide.
source
# SpaceLiDAR.ICESat2_Granule \u2014 Type.
ICESat2_Granule{product} <: Granule\n
A granule of the ICESat-2 product product. Normally created automatically from either find, granule_from_file or granules_from_folder.
source
# Base.convert \u2014 Method.
Base.convert(product::Symbol, g::ICESat2_Granule{T})\n
Converts the granule g to the product product, by guessing the correct name.
source
# SpaceLiDAR.bounds \u2014 Method.
bounds(granule::ICESat2_Granule)\n
Retrieves the bounding box of the granule.
Warning
This opens the .h5 file, so it is slow.
Example
julia> bounds(g)\ng = ICESat2_Granule()\n
source
# SpaceLiDAR.info \u2014 Method.
info(g::ICESat2_Granule)\n
Derive info based on the filename. The name is built up as follows: ATL03_[yyyymmdd][hhmmss]_[ttttccss]_[vvv_rr].h5. See section 1.2.5 in the user guide.
source
# SpaceLiDAR.GEDI_Granule \u2014 Type.
GEDI_Granule{product} <: Granule\n
A granule of the GEDI product product. Normally created automatically from either find, granule_from_file or granules_from_folder.
source
# SpaceLiDAR.info \u2014 Method.
info(g::GEDI_Granule)\n
Derive info based on the filename. This is built up as follows: GEDI02_A_2019110014613_O01991_T04905_02_001_01.h5 or in case of v\"2\": GEDI02_A_2019242104318_O04046_01_T02343_02_003_02_V002.h5. See section 2.4 in the user guide.
source
"},{"location":"reference/search/","title":"Search","text":""},{"location":"reference/search/#search","title":"Search","text":"# SpaceLiDAR.search \u2014 Function.
search(mission::Mission, bbox::Extent)\nsearch(:GEDI02_A, \"002\") # searches *all* GEDI v2 granules\n
Search granules for a given mission and bounding box.
source
"},{"location":"reference/utilities/","title":"Utilities","text":""},{"location":"reference/utilities/#utilities","title":"Utilities","text":"# SpaceLiDAR.angle! \u2014 Method.
angle!(table)\n
Sets the angle column in table as returned from points. See angle for details.
source
# SpaceLiDAR.shift \u2014 Method.
shift(longitude, latitude, angle, distance)\n
Shift longitude and latitude with distance in [m] in direction angle, where North is 0\u00b0. Returns a tuple of the shifted coordinates: (longitude, latitude). Useful for offsetting SpaceLiDAR points to the left or right of the track, in combination with angle.
source
# SpaceLiDAR.to_egm2008! \u2014 Method.
to_egm2008!(table)\n
Converts ellipsoid heights to geoid heights using the EGM2008 geoid model. Assumes a table as generated from points with columns :latitude, :longitude, and :height. Will overwrite the :height column with the geoid height.
source
<a id='SpaceLiDAR.instantiate-Union{Tuple{T}, Tuple{Vector{T}, AbstractString}} where T<:SpaceLiDAR.Granule' href='#SpaceLiDAR.instantiate-Union{Tuple{T}, Tuple{Vector{T}, AbstractString}} where T<:SpaceLiDAR.Granule'># SpaceLiDAR.instantiate \u2014 Method.
instantiate(granules::Vector{::Granule}, folder::AbstractString)\n
For a given list of granules from find, match the granules to the local files and return a new list of granules with the local filepaths if they exist.
source
# SpaceLiDAR.isvalid \u2014 Method.
isvalid(g::Granule)\n
Checks if a granule is has a valid, local and non-corrupt .h5 file. Can be combined with rm(::Granule) to remove invalid granules.
source
# SpaceLiDAR.netrc! \u2014 Method.
netrc!(username, password)\n
Writes/updates a .netrc file for ICESat-2 and GEDI downloads. A .netrc is a plaintext file containing your username and password for NASA EarthData and DAACs, and can be automatically used by Julia using Downloads and tools like wget, curl among others.
source
"},{"location":"topics/GEDI/","title":"GEDI","text":""},{"location":"topics/GEDI/#gedi","title":"GEDI","text":"GEDI-launched in 2018-is attached to the ISS and is to investigate global ecosystems1.
We currently only support the L2A product (at version 2).
See their website.
-
Dubayah, Ralph, James Bryan Blair, Scott Goetz, Lola Fatoyinbo, Matthew Hansen, Sean Healey, Michelle Hofton, et al. 2020. \u201cThe Global Ecosystem Dynamics Investigation: High-Resolution Laser Ranging of the Earth\u2019s Forests and Topography.\u201d Science of Remote Sensing 1 (June): 100002. https://doi.org/10/ggjxx8.\u00a0\u21a9
"},{"location":"topics/ICESat-2/","title":"ICESat-2","text":""},{"location":"topics/ICESat-2/#icesat-2","title":"ICESat-2","text":"ICESat-2 is in a polar orbit to investigate ice sheets1 using its Advanced Topographic Laser Altimeter System (ATLAS).
We currently support the ATL03, ATL06, ATL08 and ATL12 data products (at version 5).
See their website.
-
Markus, Thorsten, Tom Neumann, Anthony Martino, Waleed Abdalati, Kelly Brunt, Beata Csatho, Sinead Farrell, et al. 2017. \u201cThe Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2): Science Requirements, Concept, and Implementation.\u201d Remote Sensing of Environment 190 (March): 260\u201373. https://doi.org/10/gg3f7c.\u00a0\u21a9
"},{"location":"topics/ICESat/","title":"ICESat","text":""},{"location":"topics/ICESat/#icesat","title":"ICESat","text":"The Ice, Cloud, and land Elevation Satellite (1) (ICESat) was the first earth-based LiDAR satellite1, launched in 2003, with the primary goal of ice sheet monitoring. It had an elevation accuracy of several cm1 and was operational for five years.
We currently support the GLAH06 (Land Ice) and GLA14 (Land Surface) data products (at version 34).
See their website.
-
Schutz, B. E., H. J. Zwally, C. A. Shuman, D. Hancock, and J. P. DiMarzio. 2005. \u201cOverview of the ICESat Mission.\u201d Geophysical Research Letters 32 (21). https://doi.org/10/bzbspg.\u00a0\u21a9\u21a9
"},{"location":"tutorial/installation/","title":"Installation","text":""},{"location":"tutorial/installation/#installation","title":"Installation","text":"Use the Julia built-in package manager:
] add SpaceLiDAR\n
"},{"location":"tutorial/usage/","title":"Usage","text":""},{"location":"tutorial/usage/#usage","title":"Usage","text":""},{"location":"tutorial/usage/#search-for-data","title":"Search for data","text":"using SpaceLiDAR\n# Find all ATL08 granules\ngranules = search(:ICESat2, :ATL08)\n\n# Find only ATL03 granules in a part of Vietnam\nvietnam = Extent(X = (102., 107.0), Y = (8.0, 12.0))\ngranules = search(:ICESat2, :ATL08; extent=vietnam, version=6)\n\n# Find GEDI granules in the same way\ngranules = search(:GEDI, :GEDI02_A)\n\n# A granule is pretty simple\ngranule = granules[1]\ngranule.id # filename\ngranule.url # download url\ngranule.info # derived information from id\n\n# Downloading granules requires a setup .netrc with an NASA EarthData account\n# we provide a helper function, that creates/updates a ~/.netrc or ~/_netrc\nSpaceLiDAR.netrc!(username, password) # replace with your credentials\n\n# Afterward you can download the dataset\nfn = SpaceLiDAR.download!(granule)\n\n# You can also load a granule from disk\ngranule = granule(fn)\n\n# Or from a folder\nlocal_granules = granules(folder)\n\n# Instantiate search results locally (useful for GEDI location indexing)\nlocal_granules = instantiate(granules, folder)\n
"},{"location":"tutorial/usage/#derive-points","title":"Derive points","text":"using DataFrames\nfn = \"GEDI02_A_2019242104318_O04046_01_T02343_02_003_02_V002.h5\"\ng = SpaceLiDAR.granule(fn)\ndf = DataFrame(g)\n149680\u00d715 DataFrame\n Row \u2502 longitude latitude height height_error datetime intensity sensitivity surface quality nmo \u22ef\n \u2502 Float64 Float64 Float32 Float32 DateTime Float32 Float32 Bool Bool UIn \u22ef\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n 1 \u2502 153.855 -47.2772 -13.3536 0.307976 2019-08-30T10:48:21.047 393.969 -0.0671094 true false \u22ef\n 2 \u2502 153.855 -47.2769 -11.2522 0.307978 2019-08-30T10:48:21.055 797.26 0.533529 true true\n 3 \u2502 153.856 -47.2767 -13.775 0.307981 2019-08-30T10:48:21.063 1010.39 0.695938 true true\n 4 \u2502 153.857 -47.2765 -11.729 0.307983 2019-08-30T10:48:21.071 852.614 0.544849 true true\n 5 \u2502 153.857 -47.2763 -13.2443 0.307985 2019-08-30T10:48:21.080 980.66 0.620767 true true \u22ef\n 6 \u2502 153.858 -47.2761 -12.1813 0.307987 2019-08-30T10:48:21.088 937.441 0.620531 true true\n 7 \u2502 153.859 -47.2758 -11.9011 0.30799 2019-08-30T10:48:21.096 1235.02 0.73815 true true\n 8 \u2502 153.859 -47.2756 -12.3796 0.307992 2019-08-30T10:48:21.104 854.127 0.545655 true true\n
"},{"location":"tutorial/usage/#derive-linestrings","title":"Derive linestrings","text":"using DataFrames\nfn = \"ATL03_20181110072251_06520101_003_01.h5\"\ng = SpaceLiDAR.granule(fn)\ntlines = DataFrame(SpaceLiDAR.lines(g, step=10000))\nTable with 4 columns and 6 rows:\n geom sun_angle track datetime\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n 1 \u2502 wkbLineString25D geometry 38.3864 gt1l_weak 2018-11-10T07:28:01.688\n 2 \u2502 wkbLineString25D geometry 38.375 gt1r_strong 2018-11-10T07:28:02.266\n 3 \u2502 wkbLineString25D geometry 38.2487 gt2l_weak 2018-11-10T07:28:04.474\n 4 \u2502 wkbLineString25D geometry 38.1424 gt2r_strong 2018-11-10T07:28:07.374\n 5 \u2502 wkbLineString25D geometry 38.2016 gt3l_weak 2018-11-10T07:28:05.051\n 6 \u2502 wkbLineString25D geometry 38.1611 gt3r_strong 2018-11-10T07:28:06.344\nSpaceLiDAR.GDF.write(\"lines.gpkg\", tlines)\n
"}]}
\ No newline at end of file
+{"config":{"lang":["en"],"separator":"[\\s\\-]+","pipeline":["stopWordFilter"]},"docs":[{"location":"","title":"Home","text":""},{"location":"#spacelidar","title":"SpaceLiDAR","text":"A Julia toolbox for ICESat, ICESat-2 and GEDI data. Quickly search, download and load filtered point data with relevant attributes from the .h5 granules of each data product. For an overview with code examples, see the FOSS4G Pluto notebook here.
If you use SpaceLiDAR in your research, please consider citing it.
"},{"location":"#supported-data-products","title":"Supported data products","text":"Currently supports the following data products:
mission data product User Guide (UG) Algorithm Theoretical Basis Document (ATBD) ICESat GLAH06 v34 UG ATBD ICESat GLAH14 v34 UG ATBD ICESat-2 ATL03 v6 UG ATBD ICESat-2 ATL06 v5 UG ATBD ICESat-2 ATL08 v6 UG ATBD ICESat-2 ATL12 v5 UG ATBD GEDI L2A v2 UG ATBD "},{"location":"#documentation","title":"Documentation","text":"This documentation is set up using the Divio documentation system and thus split into Tutorials, Guides, Topics and References.
"},{"location":"#publications","title":"Publications","text":"The code produced for the following paper was the beginning of this package:
Vernimmen, Ronald, Aljosja Hooijer, and Maarten Pronk. 2020. \u2018New ICESat-2 Satellite LiDAR Data Allow First Global Lowland DTM Suitable for Accurate Coastal Flood Risk Assessment\u2019. Remote Sensing 12 (17): 2827. https://doi.org/10/gg9dg6.
The DTM produced using ICESat-2 ATL08 data was in turn used for:
Hooijer, A., and R. Vernimmen. 2021. \u2018Global LiDAR Land Elevation Data Reveal Greatest Sea-Level Rise Vulnerability in the Tropics\u2019. Nature Communications 12 (1): 3592. https://doi.org/10/gkzf49.
"},{"location":"changelog/","title":"Changelog","text":""},{"location":"changelog/#unreleased","title":"Unreleased","text":" - Working on generic retrieval of parameters as a Table from a granule, instead of the hardcoded choices made now for each product. Would result in methods like
points(granule, [vara, varb]).
"},{"location":"changelog/#new-features","title":"New features","text":" - New types
Table and PartitionedTable, which support the Tables.jl interface. This prevents allocating code like reduce(vcat, DataFrame.(points(granule))) to get a DataFrame. You can now just call DataFrame(table). - Reduced allocations in retrieving point data.
- Introduced
before and after keywords in search, to search by date(ranges).
"},{"location":"changelog/#fixed","title":"Fixed","text":" - Empty (filtered) granules could result in
Vector{BitVector} columns, which have been changed to Vector{Bool}.
"},{"location":"changelog/#breaking","title":"Breaking","text":" points now return either a Table or a PartitionedTable instead of NamedTuple or Vector{NamedTuple}. The old behaviour can be regained by calling parent on these tables.- Removed
number_of_returns and return_number from ICESat-2 ATL08 canopy output.
"},{"location":"changelog/#deprecated","title":"Deprecated","text":" - Renamed
granule_from_file to granule - Renamed
granules_from_file to granules - Renamed
write_granule_urls! to write_urls
"},{"location":"changelog/#changed","title":"Changed","text":" - Most of the search functionality has been moved out to the more generic EarthData.jl
- Updated ICESat-2 from version 5 to version 6
"},{"location":"changelog/#v030","title":"v0.3.0","text":" - GeoInterface, Extents support
- Bounding box using Extent subsetting on all
points functions find renamed to searchsearch now takes the product as a Symbol and accepts a s3::Bool parameter.- Stricter checking of arguments in
search - MultiPolygon extents of granules are now returned in the
polygons field in granules from search. - Removed
bbox field on granules. download(granule) now works on AWS S3 urlsdonwload(granule) now creates temporary files and renames after successful download- ICESat-2 ATL08 v5
points function now supports 20 m resolution by passing highres::Bool parameter.
"},{"location":"changelog/#v022","title":"v0.2.2","text":" - Fixed ICESat-2 download (please remove
n5eil01u.ecs.nsidc.org from your .netrc file) - Linked to Zenodo for DOI citations
"},{"location":"changelog/#v021","title":"v0.2.1","text":" - Unified
bounds of granules - Fixed
getcoord for Point and added Point(x, y, z) constructor - Updated utils to make use of DataFrames with correct column names
"},{"location":"changelog/#v020","title":"v0.2.0","text":"Warning
This is a breaking release
- Many of the column names have changed to be more descriptive.
- Documentation and docstring improvements.
- Tables support, you can now do
DataFrame(granule), without having to call points(granule). - Memory use improvements, by using SentinelArray of FillArray under the hood.
- Dropped S3, GeoArrays and LAS/LAZ support.
- Added GeoInterface support for lines/points and dropped GeoDataFrames
- Expanded test coverage.
"},{"location":"changelog/#v016","title":"v0.1.6","text":" - Support for ICESat GLAH06 by alex-s-gardner
"},{"location":"changelog/#v015","title":"v0.1.5","text":" - Support for ICESat-2 ATL06
- Update search to use v5 for ICESat-2 by default
"},{"location":"changelog/#v014","title":"v0.1.4","text":" - Compatibility fixes
"},{"location":"changelog/#v013","title":"v0.1.3","text":" - Added interpolation for GeoArrays
- Added FOSS4G notebook
"},{"location":"guides/downloads/","title":"Downloading","text":""},{"location":"guides/downloads/#downloading","title":"Downloading","text":"As shown in Tutorial: Search for data, it is easy to find data. Downloading the data might be harder though, especially when downloading a large amount of granules or even mirroring a complete DAAC. Indeed, the Julia download! won't work in parallel, nor will it resume downloads or show its progress. In such cases it's useful to export a list of granules to a text file and use an external download tool:
granules = find(:ICESat2, \"ATL08\")\nSpaceLiDAR.write_urls(\"atl08_world.txt\", granules)\n
In my case, I use aria2c. Note that downloading from the granule urls require a EarthData login, normally setup in an .netrc file (also see netrc!).
aria2c -c -i atl08_world.txt\n
Once finished, one can again instantiate the list of granules with the folder to which all files have been downloaded.
"},{"location":"guides/tracks/","title":"Track filtering","text":""},{"location":"guides/tracks/#track-filtering","title":"Track filtering","text":"Coming soon
"},{"location":"reference/api/","title":"Index","text":""},{"location":"reference/api/#api-reference","title":"API reference","text":" SpaceLiDAR.GEDI_GranuleSpaceLiDAR.ICESat2_GranuleSpaceLiDAR.ICESat_GranuleBase.Filesystem.rmBase.convertSpaceLiDAR.angle!SpaceLiDAR.boundsSpaceLiDAR.boundsSpaceLiDAR.classifySpaceLiDAR.downloadSpaceLiDAR.downloadSpaceLiDAR.download!SpaceLiDAR.download!SpaceLiDAR.infoSpaceLiDAR.infoSpaceLiDAR.infoSpaceLiDAR.instantiateSpaceLiDAR.isvalidSpaceLiDAR.netrc!SpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.pointsSpaceLiDAR.searchSpaceLiDAR.shiftSpaceLiDAR.syncSpaceLiDAR.to_egm2008!
"},{"location":"reference/data/","title":"Data","text":""},{"location":"reference/data/#data","title":"Data","text":""},{"location":"reference/data/#icesat","title":"ICESat","text":"# SpaceLiDAR.points \u2014 Method.
points(g::ICESat_Granule{:GLAH06}, step=1, bbox::Union{Nothing,Extent,NamedTuple} = nothing)\n
Retrieve the points for a given ICESat GLAH06 (Land Ice) granule as a list of namedtuples The names of the tuples are based on the following fields:
Variable Original Field Description Units longitude Data_40HZ/Geolocation/d_lon Longitude of segment center, WGS84, East=+ decimal degrees latitude Data_40HZ/Geolocation/d_lat Latitude of segment center, WGS84, North=+ decimal degrees height Data_40HZ/Elevation_Surfaces/d_elev + Data_40HZ/Elevation_Corrections/d_satElevCorr m above WGS84 ellipsoid datetime Data_40HZ/DS_UTCTime_40 Precise time of aquisiton date-time quality 1 Data_40HZ/Quality/elev_use_flg & Data_40HZ/Quality/sigma_att_flg = 0 & Data_40HZ/Waveform/i_numPk = 1 & Data_40HZ/Elevation_Corrections/d_satElevCorr < 3 1 = high quality height_reference land_ice_segments/dem/dem_h Height of the (best available) DEM height above WGS84 You can get the output in a DataFrame with DataFrame(points(g)).
source
# SpaceLiDAR.points \u2014 Method.
points(g::ICESat_Granule{:GLAH14}, step=1, bbox::Union{Nothing,Extent,NamedTuple} = nothing)\n
Retrieve the points for a given ICESat GLAH14 (Land Surface) granule as a list of namedtuples The names of the tuples are based on the following fields:
Variable Original Field (in Data_40HZ) Description Units longitude /Geolocation/d_lon Longitude of segment center, WGS84, East=+ decimal degrees latitude Geolocation/d_lat Latitude of segment center, WGS84, North=+ decimal degrees height Elevation_Surfaces/d_elev + Elevation_Corrections/d_satElevCorr m above WGS84 ellipsoid datetime DS_UTCTime_40 Precise time of aquisiton date-time quality 1 Quality/elev_use_flg & Quality/sigma_att_flg = 0 & Waveform/i_numPk = 1 & Elevation_Corrections/d_satElevCorr < 3 1=high quality clouds Elevation_Flags/elv_cloud_flg Cloud contamination - height_reference Geophysical/d_DEM_elv Height of the (best available) DEM height above WGS84 gain Waveform/i_gval_rcv Gain value used for received pulse. - reflectivity Reflectivity/d_reflctUC Reflectivity, not corrected - attitude Quality/sigma_att_flg Attitude quality indicator 0=good; 50=warning; 100=bad; saturation Quality/sat_corr_flg Saturation Correction Flag 0=not_saturated; You can get the output in a DataFrame with DataFrame(points(g)).
source
"},{"location":"reference/data/#icesat-2","title":"ICESat-2","text":"# SpaceLiDAR.classify \u2014 Function.
classify(granule::ICESat2_Granule{:ATL03}, atl08::Union{ICESat2_Granule{:ATL08},Nothing} = nothing, tracks = icesat2_tracks)\n
Like points(::ICESat2_Granule{:ATL03}) but with the classification from the ATL08 dataset. If an ATL08 granule is not provided, we try to find it based on the ATL03 name using convert.
source
# SpaceLiDAR.points \u2014 Method.
points(g::ICESat2_Granule{:ATL03}, tracks=icesat2_tracks; step=1, bbox::Union{Nothing,Extent,NamedTuple} = nothing)\n
Retrieve the points for a given ICESat-2 ATL03 (Global Geolocated Photon Data) granule as a list of namedtuples, one for each beam. The names of the tuples are based on the following fields:
Column Field Description Units longitude heights/lon_ph Longitude of photon, WGS84, East=+ decimal degrees latitude heights/lat_ph Latitude of photon, WGS84, North=+ decimal degrees height heights/h_ph Photon WGS84 Height m above the WGS 84 ellipsoid quality heights/quality_ph Indicates the quality of the associated photon 0 = nominal uncertainty geolocation/sigma_h Estimated height uncertainty m datetime heights/delta_time + ancillary_data/atlas_sdp_gps_epoch + gps_offset date-time confidence heights/signal_conf_ph Photon Signal Confidence 2=low; 3=med; 4=high segment geolocation/segment_id Along-track segment ID number - track gt1l - gt3r groups - - strong_beam - \"strong\" (true) or \"weak\" (false) laser power - sun_angle geolocation/solar_elevation Sun angle \u00b0 above horizon detector_id atlas_spot_number attribute - - height_reference heights/dem/dem_h Height of the (best available) DEM m above the WGS 84 ellipsoid You can combine the output in a DataFrame with reduce(vcat, DataFrame.(points(g))) if you want to change the default arguments or DataFrame(g) with the default options.
source
# SpaceLiDAR.points \u2014 Method.
points(g::ICESat2_Granule{:ATL06}, tracks=icesat2_tracks, step=1, bbox::Union{Nothing,Extent,NamedTuple} = nothing)\n
Retrieve the points for a given ICESat-2 ATL06 (Land Ice) granule as a list of namedtuples, one for each beam. The names of the tuples are based on the following fields:
Column Field Description Units longitude land_ice_segments/longitude Longitude of segment center, WGS84, East=+ decimal degrees latitude land_ice_segments/latitude Latitude of segment center, WGS84, North=+ decimal degrees height land_ice_segments/h_li Standard land-ice segment height m above the WGS 84 ellipsoid height_error \u221a(land_ice_segments/sigma_geo_h\u00b2 + Total vertical geolocation error m above the WGS 84 ellipsoid land_ice_segments/h_li_sigma\u00b2) datetime land_ice_segments/delta_time + ancillary_data/atlas_sdp_gps_epoch + gps_offset date-time quality land_ice_segments/atl06_quality_summary Boolean flag indicating the best-quality subset 1 = high quality track gt1l - gt3r groups - - strong_beam - \"strong\" (true) or \"weak\" (false) laser power - detector_id atlas_spot_number attribute - - height_reference land_ice_segments/dem/dem_h Height of the (best available) DEM - You can combine the output in a DataFrame with reduce(vcat, DataFrame.(points(g))) if you want to change the default arguments or DataFrame(g) with the default options.
source
# SpaceLiDAR.points \u2014 Method.
points(g::ICESat2_Granule{:ATL08}; tracks=icesat2_tracks, step=1, canopy=false, ground=true, bbox::Union{Nothing,Extent,NamedTuple} = nothing)\n
Retrieve the points for a given ICESat-2 ATL08 (Land and Vegetation Height) granule as a list of namedtuples, one for each beam. With the tracks keyword, you can specify which tracks to include. The default is to include all tracks. With the step keyword, you can choose to limit the number of points, the default is 1 (all points). With setting ground and or canopy, you can control to include ground and/or canopy points. Finally, with the ground_field and canopy_field settings, you can determine the source field. The default is h_te_mean for ground and h_mean_canopy_abs for canopy. With the introduction of v5, a 20m resolution is also available for estimation, which you can enable with highres. Note that filtering with a bounding box doesn't yet work when highres is true.
The names of the tuples are based on the following fields:
Column Field Description Units longitude land_segments/longitude Longitude of segment center, WGS84, East=+ decimal degrees latitude land_segments/latitude Latitude of segment center, WGS84, North=+ decimal degrees height land_segments/terrain/h_te_mean Standard land-ice segment height m above the WGS 84 ellipsoid height_error land_segments/terrain/h_te_uncertainty Total vertical geolocation error m datetime land_segments/delta_time + ancillary_data/atlas_sdp_gps_epoch + gps_offset date-time quality land_segments/terrain_flg Boolean flag indicating the best-quality subset 1 = high quality phr land_segments/ph_removal_flag More than 50% of photons removed - sensitivity land_segments/snr The signal to noise ratio - scattered land_segments/msw_flag Multiple Scattering warning flag -1=unknown; 0=none saturated land_segments/sat_flag Saturation detected - clouds land_segments/layer_flag Clouds or blowing snow are likely present - track gt1l - gt3r groups - - strong_beam - \"strong\" (true) or \"weak\" (false) laser power - classification - \"ground\", \"high_canopy\" - height_reference land_segments/dem_h Height of the (best available) DEM m above the WGS 84 ellipsoid detector_id atlas_spot_number attribute - - You can combine the output in a DataFrame with reduce(vcat, DataFrame.(points(g))) if you want to change the default arguments or just DataFrame(g) with the default options.
source
# SpaceLiDAR.points \u2014 Function.
points(g::ICESat2_Granule{:ATL12}, tracks=icesat2_tracks)\n
Retrieve the points for a given ICESat-2 ATL12 (Ocean Surface Height) granule as a list of namedtuples, one for each beam. The names of the tuples are based on the following fields:
Column Field Description Units longitude ssh_segments/longitude Longitude of segment center, WGS84, East=+ decimal degrees latitude ssh_segments/latitude Latitude of segment center, WGS84, North=+ decimal degrees height ssh_segments/heights/h Standard land-ice segment height m above the WGS 84 ellipsoid datetime ssh_segments/delta_time + ancillary_data/atlas_sdp_gps_epoch + gps_offset date-time track gt1l - gt3r groups - - strong_beam - \"strong\" (true) or \"weak\" (false) laser power - detector_id atlas_spot_number attribute - - You can combine the output in a DataFrame with reduce(vcat, DataFrame.(points(g))) if you want to change the default arguments or DataFrame(g) with the default options.
source
"},{"location":"reference/data/#gedi","title":"GEDI","text":"# SpaceLiDAR.bounds \u2014 Method.
bounds(granule::GEDI_Granule)\n
Return the bounds of the GEDI granule.
Warning
This opens the .h5 file to read all tracks, so it is very slow.
source
-
Smith, B., Fricker, H. A., Gardner, A. S., Medley, B., Nilsson, J., Paolo, F. S., ... & Zwally, H. J. (2020). Pervasive ice sheet mass loss reflects competing ocean and atmosphere processes. Science, 368(6496), 1239-1242.\u00a0\u21a9\u21a9
"},{"location":"reference/granules/","title":"Granules","text":""},{"location":"reference/granules/#granules","title":"Granules","text":"# Base.Filesystem.rm \u2014 Method.
rm(granule::Granule)\n
Remove the file associated with granule from the local filesystem.
source
# SpaceLiDAR.download \u2014 Function.
download(granules::Vector{<:Granule}, folder=\".\")\n
Like download, but for a vector of granules.
source
# SpaceLiDAR.download \u2014 Function.
download(granule::Granule, folder=\".\")\n
Download the file associated with granule to the folder, from an http(s) location if it doesn't already exists locally. Returns a new granule. See download! for a mutating version.
Will require credentials (netrc) which can be set with netrc!.
source
# SpaceLiDAR.download! \u2014 Function.
download!(granule::Granule, folder=\".\")\n
Download the file associated with granule to the folder, from an http(s) location if it doesn't already exists locally.
Will require credentials (netrc) which can be set with netrc!.
source
# SpaceLiDAR.download! \u2014 Function.
download!(granules::Vector{<:Granule}, folder=\".\")\n
Like download!, but for a vector of granules. Will make use of aria2c (parallel).
source
# SpaceLiDAR.sync \u2014 Function.
sync(folder::AbstractString, all::Bool=false; kwargs...)\nsync(folders::AbstractVector{<:AbstractString}, all::Bool=false; kwargs...)\nsync(product::Symbol, folder::AbstractString, all::Bool=false; kwargs...)\nsync(product::Symbol, folders::AbstractVector{<:AbstractString}, all::Bool=false; kwargs...)\n
Syncronize an existing archive of local granules in folder(s) with the latest granules available. Specifically, this will run search and download for any granules not yet present in folder(s), to the first folder in the list.
Warning
Using sync could result in downloading significant (TB+) amounts of data.
Assumes all folders contain granules of the same product. If not, pass the product as Symbol: sync(::Symbol, folders, all) instead.
When all is false (the default), sync will search only for granules past the date of the latest granule found in folders. If true, it will search for all granules. Note that ICESat granules are not timestamped, so sync will try to download all ICESat granules not yet present, regardless of this setting.
Any kwargs... are passed to the search function. This enables sync to only download granules within a certain extent, for example.
source
# SpaceLiDAR.ICESat_Granule \u2014 Type.
ICESat_Granule{product} <: Granule\n
A granule of the ICESat product product. Normally created automatically from either find, granule_from_file or granules_from_folder.
source
# SpaceLiDAR.info \u2014 Method.
info(g::ICESat_Granule)\n
Derive info based on the filename. The name is built up as follows: ATL03yyyymmdd[ttttccss][vvvrr].h5. See section 1.2.5 in the user guide.
source
# SpaceLiDAR.ICESat2_Granule \u2014 Type.
ICESat2_Granule{product} <: Granule\n
A granule of the ICESat-2 product product. Normally created automatically from either find, granule_from_file or granules_from_folder.
source
# Base.convert \u2014 Method.
Base.convert(product::Symbol, g::ICESat2_Granule{T})\n
Converts the granule g to the product product, by guessing the correct name.
source
# SpaceLiDAR.bounds \u2014 Method.
bounds(granule::ICESat2_Granule)\n
Retrieves the bounding box of the granule.
Warning
This opens the .h5 file, so it is slow.
Example
julia> bounds(g)\ng = ICESat2_Granule()\n
source
# SpaceLiDAR.info \u2014 Method.
info(g::ICESat2_Granule)\n
Derive info based on the filename. The name is built up as follows: ATL03_[yyyymmdd][hhmmss]_[ttttccss]_[vvv_rr].h5. See section 1.2.5 in the user guide.
source
# SpaceLiDAR.GEDI_Granule \u2014 Type.
GEDI_Granule{product} <: Granule\n
A granule of the GEDI product product. Normally created automatically from either find, granule_from_file or granules_from_folder.
source
# SpaceLiDAR.info \u2014 Method.
info(g::GEDI_Granule)\n
Derive info based on the filename. This is built up as follows: GEDI02_A_2019110014613_O01991_T04905_02_001_01.h5 or in case of v\"2\": GEDI02_A_2019242104318_O04046_01_T02343_02_003_02_V002.h5. See section 2.4 in the user guide.
source
"},{"location":"reference/search/","title":"Search","text":""},{"location":"reference/search/#search","title":"Search","text":"# SpaceLiDAR.search \u2014 Function.
search(mission::Mission, bbox::Extent)\nsearch(:GEDI02_A, \"002\") # searches *all* GEDI v2 granules\n
Search granules for a given mission and bounding box.
source
"},{"location":"reference/utilities/","title":"Utilities","text":""},{"location":"reference/utilities/#utilities","title":"Utilities","text":"# SpaceLiDAR.angle! \u2014 Method.
angle!(table)\n
Sets the angle column in table as returned from points. See angle for details.
source
# SpaceLiDAR.shift \u2014 Method.
shift(longitude, latitude, angle, distance)\n
Shift longitude and latitude with distance in [m] in direction angle, where North is 0\u00b0. Returns a tuple of the shifted coordinates: (longitude, latitude). Useful for offsetting SpaceLiDAR points to the left or right of the track, in combination with angle.
source
# SpaceLiDAR.to_egm2008! \u2014 Method.
to_egm2008!(table)\n
Converts ellipsoid heights to geoid heights using the EGM2008 geoid model. Assumes a table as generated from points with columns :latitude, :longitude, and :height. Will overwrite the :height column with the geoid height.
source
<a id='SpaceLiDAR.instantiate-Union{Tuple{T}, Tuple{Vector{T}, AbstractString}} where T<:SpaceLiDAR.Granule' href='#SpaceLiDAR.instantiate-Union{Tuple{T}, Tuple{Vector{T}, AbstractString}} where T<:SpaceLiDAR.Granule'># SpaceLiDAR.instantiate \u2014 Method.
instantiate(granules::Vector{::Granule}, folder::AbstractString)\n
For a given list of granules from find, match the granules to the local files and return a new list of granules with the local filepaths if they exist.
source
# SpaceLiDAR.isvalid \u2014 Method.
isvalid(g::Granule)\n
Checks if a granule is has a valid, local and non-corrupt .h5 file. Can be combined with rm(::Granule) to remove invalid granules.
source
# SpaceLiDAR.netrc! \u2014 Method.
netrc!(username, password)\n
Writes/updates a .netrc file for ICESat-2 and GEDI downloads. A .netrc is a plaintext file containing your username and password for NASA EarthData and DAACs, and can be automatically used by Julia using Downloads and tools like wget, curl among others.
source
"},{"location":"topics/GEDI/","title":"GEDI","text":""},{"location":"topics/GEDI/#gedi","title":"GEDI","text":"GEDI-launched in 2018-is attached to the ISS and is to investigate global ecosystems1.
We currently only support the L2A product (at version 2).
See their website.
-
Dubayah, Ralph, James Bryan Blair, Scott Goetz, Lola Fatoyinbo, Matthew Hansen, Sean Healey, Michelle Hofton, et al. 2020. \u201cThe Global Ecosystem Dynamics Investigation: High-Resolution Laser Ranging of the Earth\u2019s Forests and Topography.\u201d Science of Remote Sensing 1 (June): 100002. https://doi.org/10/ggjxx8.\u00a0\u21a9
"},{"location":"topics/ICESat-2/","title":"ICESat-2","text":""},{"location":"topics/ICESat-2/#icesat-2","title":"ICESat-2","text":"ICESat-2 is in a polar orbit to investigate ice sheets1 using its Advanced Topographic Laser Altimeter System (ATLAS).
We currently support the ATL03, ATL06, ATL08 and ATL12 data products (at version 5).
See their website.
-
Markus, Thorsten, Tom Neumann, Anthony Martino, Waleed Abdalati, Kelly Brunt, Beata Csatho, Sinead Farrell, et al. 2017. \u201cThe Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2): Science Requirements, Concept, and Implementation.\u201d Remote Sensing of Environment 190 (March): 260\u201373. https://doi.org/10/gg3f7c.\u00a0\u21a9
"},{"location":"topics/ICESat/","title":"ICESat","text":""},{"location":"topics/ICESat/#icesat","title":"ICESat","text":"The Ice, Cloud, and land Elevation Satellite (1) (ICESat) was the first earth-based LiDAR satellite1, launched in 2003, with the primary goal of ice sheet monitoring. It had an elevation accuracy of several cm1 and was operational for five years.
We currently support the GLAH06 (Land Ice) and GLA14 (Land Surface) data products (at version 34).
See their website.
-
Schutz, B. E., H. J. Zwally, C. A. Shuman, D. Hancock, and J. P. DiMarzio. 2005. \u201cOverview of the ICESat Mission.\u201d Geophysical Research Letters 32 (21). https://doi.org/10/bzbspg.\u00a0\u21a9\u21a9
"},{"location":"tutorial/installation/","title":"Installation","text":""},{"location":"tutorial/installation/#installation","title":"Installation","text":"Use the Julia built-in package manager:
] add SpaceLiDAR\n
"},{"location":"tutorial/usage/","title":"Usage","text":""},{"location":"tutorial/usage/#usage","title":"Usage","text":""},{"location":"tutorial/usage/#search-for-data","title":"Search for data","text":"using SpaceLiDAR\n# Find all ATL08 granules\ngranules = search(:ICESat2, :ATL08)\n\n# Find only ATL03 granules in a part of Vietnam\nvietnam = Extent(X = (102., 107.0), Y = (8.0, 12.0))\ngranules = search(:ICESat2, :ATL08; extent=vietnam, version=6)\n\n# Find GEDI granules in the same way\ngranules = search(:GEDI, :GEDI02_A)\n\n# A granule is pretty simple\ngranule = granules[1]\ngranule.id # filename\ngranule.url # download url\ngranule.info # derived information from id\n\n# Downloading granules requires a setup .netrc with an NASA EarthData account\n# we provide a helper function, that creates/updates a ~/.netrc or ~/_netrc\nSpaceLiDAR.netrc!(username, password) # replace with your credentials\n\n# Afterward you can download the dataset\nfn = SpaceLiDAR.download!(granule)\n\n# You can also load a granule from disk\ngranule = granule(fn)\n\n# Or from a folder\nlocal_granules = granules(folder)\n\n# Instantiate search results locally (useful for GEDI location indexing)\nlocal_granules = instantiate(granules, folder)\n
"},{"location":"tutorial/usage/#derive-points","title":"Derive points","text":"using DataFrames\nfn = \"GEDI02_A_2019242104318_O04046_01_T02343_02_003_02_V002.h5\"\ng = SpaceLiDAR.granule(fn)\ndf = DataFrame(g)\n149680\u00d715 DataFrame\n Row \u2502 longitude latitude height height_error datetime intensity sensitivity surface quality nmo \u22ef\n \u2502 Float64 Float64 Float32 Float32 DateTime Float32 Float32 Bool Bool UIn \u22ef\n\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n 1 \u2502 153.855 -47.2772 -13.3536 0.307976 2019-08-30T10:48:21.047 393.969 -0.0671094 true false \u22ef\n 2 \u2502 153.855 -47.2769 -11.2522 0.307978 2019-08-30T10:48:21.055 797.26 0.533529 true true\n 3 \u2502 153.856 -47.2767 -13.775 0.307981 2019-08-30T10:48:21.063 1010.39 0.695938 true true\n 4 \u2502 153.857 -47.2765 -11.729 0.307983 2019-08-30T10:48:21.071 852.614 0.544849 true true\n 5 \u2502 153.857 -47.2763 -13.2443 0.307985 2019-08-30T10:48:21.080 980.66 0.620767 true true \u22ef\n 6 \u2502 153.858 -47.2761 -12.1813 0.307987 2019-08-30T10:48:21.088 937.441 0.620531 true true\n 7 \u2502 153.859 -47.2758 -11.9011 0.30799 2019-08-30T10:48:21.096 1235.02 0.73815 true true\n 8 \u2502 153.859 -47.2756 -12.3796 0.307992 2019-08-30T10:48:21.104 854.127 0.545655 true true\n
"},{"location":"tutorial/usage/#derive-linestrings","title":"Derive linestrings","text":"using DataFrames\nfn = \"ATL03_20181110072251_06520101_003_01.h5\"\ng = SpaceLiDAR.granule(fn)\ntlines = DataFrame(SpaceLiDAR.lines(g, step=10000))\nTable with 4 columns and 6 rows:\n geom sun_angle track datetime\n \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\n 1 \u2502 wkbLineString25D geometry 38.3864 gt1l_weak 2018-11-10T07:28:01.688\n 2 \u2502 wkbLineString25D geometry 38.375 gt1r_strong 2018-11-10T07:28:02.266\n 3 \u2502 wkbLineString25D geometry 38.2487 gt2l_weak 2018-11-10T07:28:04.474\n 4 \u2502 wkbLineString25D geometry 38.1424 gt2r_strong 2018-11-10T07:28:07.374\n 5 \u2502 wkbLineString25D geometry 38.2016 gt3l_weak 2018-11-10T07:28:05.051\n 6 \u2502 wkbLineString25D geometry 38.1611 gt3r_strong 2018-11-10T07:28:06.344\nSpaceLiDAR.GDF.write(\"lines.gpkg\", tlines)\n
"}]}
\ No newline at end of file