Skip to content

Commit

Permalink
docs: update focused on system setup, deploy, upgrade, and contributi…
Browse files Browse the repository at this point in the history
…ng (#394)

* upgrade: update upgrade steps and migration

* Rearrange Contributing page, add WSL distribution steps

* docs: update WSL gp image publish, install, and upgrade steps.  update ubuntu and node versions

* update Box Drive instructions

* MetricGroup example cleanup and link

* docs: update deploy steps, specifically awscli

* docs: new project mention of working with datasets bucket

* docs: contributing
  • Loading branch information
twelch authored Dec 24, 2024
1 parent 685ad92 commit dd1c40a
Show file tree
Hide file tree
Showing 8 changed files with 262 additions and 162 deletions.
155 changes: 100 additions & 55 deletions website/docs/Contributing.md

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions website/docs/_components/NodeVersion.tsx
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import React from "react";

export function NodeVersion() {
return <span>20.12</span>;
return <span>22.12.0</span>;
}

export function UbuntuVersion() {
return <span>22.04 Jammy</span>;
return <span>24.04 Noble</span>;
}
2 changes: 1 addition & 1 deletion website/docs/concepts/Concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -480,7 +480,7 @@ variance?: Nullable<number>;
**/
```

### Metric Group
## Metric Group

A [MetricGroup](https://github.com/seasketch/geoprocessing/blob/dev/packages/geoprocessing/src/types/metricGroup.ts#L11) defines a specific type of metric for your project, with a base metric type, and one or more data classes. Think of it as a unifying configuration object, that pulls everything together.

Expand Down
17 changes: 8 additions & 9 deletions website/docs/tutorials/Tutorials.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,18 +107,18 @@ mkdir C:\tmp

- Download the most recent `geoprocessing-workspace` zip file from the [SeaSketch Box folder](https://ucsb.box.com/s/k9477fqzzn0yel5kf5kj2y81tst09f4i) to this tmp directory and then unzip it.

- Create a second directory to import the Geoprocessing tar image too, then import it:
- Create a second directory and import the distribution:

```bash
mkdir C:\WslDistributions\Geoprocessing
wsl --import Geoprocessing C:\WslDistributions\Geoprocessing\ C:\tmp\geoprocessing-workspace_20230627\geoprocessing-workspace_20230627_65bd30ba63a3.tar
mkdir C:\WslDistributions\gp-stable-20241223
wsl --import "gp-stable-20241223" C:\WslDistributions\gp-stable-20241223 C:\tmp\gp-stable_20241223.tar
```

- Be sure to update the filename in the import command above to match the version you downloaded.
- Be sure to update the filename in the import command above to match your tar file.
- If import is started correctly, you will see the message `Import in progress, this may take a few minutes...`. Once done it should say `The operation completed successfully`. If it didn't import successfully, try restarting your system, WSL may not have been running properly.

- Setup Terminal Profile
- This will create a shortcut to start an instance of Geoprocessing in WSL.
- This will create a shortcut to start an instance of your geoprocessing workspace image.
- In PowerShell, click the down arrow in the tab bar, to the right of the (+) icon, then click Settings.

![Terminal Profile](assets/terminal-profile.jpg "Terminal Profile")
Expand All @@ -128,10 +128,9 @@ wsl --import Geoprocessing C:\WslDistributions\Geoprocessing\ C:\tmp\geoprocessi

![Profile Duplicate](assets/terminal-duplicate.jpg "Profile Duplicate")

- Change the name of the duplicated Ubuntu profiles to `Geoprocessing`
- Change the profiles Terminal command from `C:\WINDOWS\system32\wsl.exe -d Ubuntu` to `C:\WINDOWS\system32\wsl.exe -u vscode -d Geoprocessing`. This will ensure that Geoprocessing starts with the correct user, instead of root.
- Change the profiles Terminal command from `C:\WINDOWS\system32\wsl.exe -d Ubuntu` to `C:\WINDOWS\system32\wsl.exe -u vscode -d gp-stable-20241223`. This will ensure that Geoprocessing starts with the correct image (`gp-stable-20241223`), and the right user (`vscode`).
- Save and exit your new profile
- The Terminal dropdown menu should now have a new `Geoprocessing` choice. Click this to start an instance of the Geoprocessing Distribution. It will open a shell, logged in with the vscode user.
- The Terminal dropdown menu should now have a new `gp-stable-20241223` choice. Click this to start an instance of the Geoprocessing Distribution. It will open a shell, logged in with the vscode user.

In the Geoprocessing shell, setup the workspaces directory where you will create projects:

Expand Down Expand Up @@ -162,7 +161,7 @@ ls /mnt/c

Follow the [final configuration steps](#final-configuration---all-install-options) below, then move on to creating a new project.

[Upgrade](./upgrade.md#upgrade-windows-geoprocessing) steps for the Geoprocessing Distribution are available.
[Upgrade](./upgrade.md#windows-wsl-geoprocessing-distribution) steps for the Geoprocessing Distribution are available.

### Default Ubuntu Distribution

Expand Down
18 changes: 12 additions & 6 deletions website/docs/tutorials/clouddrive.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Cloud Drive Syncing

## Box Sync
## Box Drive Sync

This is useful if you manage your spatial data in Box and collaborators sync it to their local computers. This lets you bring your Box Drive folder into your Ubuntu docker container so that you can symlink it to your `data/src` folder and import data in a consistent way across all users.
The Box Drive software application lets you manage your spatial data in a centralized Box folder, and then collaborators can sync it to their local computers. You are then able to access this Box Drive in your Docker container or WSL Geoprocessing distribution using the filesystem bridge. Specifically, you can symlink a Box subfolder to your projects `data/src` folder and access your data in a consistent way.

Note, this could be used with other drive share systems such as Google Drive.
Note, this could potentially be used with other drive share systems such as Google Drive.

First, ensure you have Box Drive installed, and you have enabled sync of your data to your local computer.

Expand All @@ -18,23 +18,29 @@ ls /Users/alex/Library/CloudStorage/Box-Box

If you see the top-level of your synced files, then you are good to go.

### Ubuntu on Windows WSL2
### Windows WSL Ubuntu

[WSL Ubuntu](./Tutorials.md#ubuntu-direct-install) seems to have the ability to access your Box drive folder over the WSL filesystem bridge. This does not seem to be true for the [WSL Geoprocessing Distribution](./Tutorials.md#geoprocessing-distribution).

Source - https://github.com/microsoft/WSL/issues/4310

Assume your box folder is installed on your Windows system at this path

```bash
C:\Users\alex\box
```

Now add the following to your .bashrc file in your home directory
Use Windows Explorer or Powershell to confirm the name of your folder.

Now add the following to your .bashrc file in the home directory of WSL Ubuntu

```bash
if ! findmnt -M /mnt/box &>/dev/null; then
sudo mount -t drvfs 'C:\Users\alex\box' /mnt/box
fi
```

On every Ubuntu shell startup you will nowbe prompted for your root password. Once entered the drive mount will be performed.
On every Ubuntu shell startup you will now be prompted for your root password. Close and reopen your Ubuntu shell. Once your password is entered the drive mount will be performed.

Now verify your box files are accessible from Linux

Expand Down
53 changes: 48 additions & 5 deletions website/docs/tutorials/deploy.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,25 +13,68 @@ For every deploy after the first, it is smart enough to compute the changeset be

## Setup AWS

You are not required to complete this step until you want to deploy your project and integrate it with SeaSketch. Until then, you can do everything except `publish` data or `deploy` your project.
You are not required to complete this step until you want to deploy your project and integrate it with SeaSketch. Until then, you can do everything **except** `publish` data or `deploy` your project.

You will need to create an AWS account with an admin user, allowing the framework to deploy your project using CloudFormation. A payment method such as a credit card will be required.

Expected cost: [free](https://aws.amazon.com/free) to a few dollars per month. You will be able to track this.

### AWS Account and User

If you are on the SeaSketch team, an AWS account and user will be provided to you.

- Create an Amazon [AWS account] such that you can login and access the main AWS Console page (https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/).
- Create an AWS IAM [admin account](https://docs.aws.amazon.com/IAM/latest/UserGuide/getting-started_create-admin-group.html). This is what you will use to manage projects.
- Setup IAM role allowing API Gateway to log info to Cloudwatch
- By default, API Gateway does not have the required permission to write logs to CloudWatch. It is necessary to specify an IAM Role. This can be accomplished by logging into the AWS console, switching to the region where you would like to deploy your geoprocessing function, and configuring this role.
- https://coady.tech/aws-cloudwatch-logs-arn/

## AWSCLI
### Install AWSCLI

If you are using a Docker devcontainer to develop reports you should already have access to the `aws` command.

But if you are developing reports directly on your host operating system you will need to install [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html) and [configure](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html) it with your IAM account credentials.

### AWSCLI Credentials - Virtual

If you are using a [virtual install](./Tutorials.md#virtual-install-with-docker-desktop) setup, it would have instructed you to create a `.env` file from the provided `.env.template` file. This is were you need to configure two environment variables with your AWS admin account credentials. When you start your devcontainer the variables will be loaded into your shell environment and picked up by all aws commands. The benefit of loading the variables this way is that they will persist and you won't have to set them every time.

```
AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY=YOUR_ACCESS_KEY
```

### AWSCLI Credentials - Windows

When running on Windows you have two options for providing your AWS credentials

1. configure credentials in Linux.
2. load credentials from Windows

#### Configure Credentials in WSL Linux

If you are using a Docker devcontaine to develop reports you should already have access to the `aws` command. But if you are running directly on your host operating system you will need to install [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html) and [configure](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html) it with your IAM account credentials.
Set environment variables in the `.bashrc` file in your Linux home directory.

```bash
nano ~/.bashrc
AWS_ACCESS_KEY_ID=YOUR_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY=YOUR_ACCESS_KEY

Ctrl-o to save
Ctrl-x to exit
```

Now, source the file into your current shell environment and verify the environment variables are set

```bash
source ~/.bashrc
echo $AWS_ACCESS_KEY_ID
echo $AWS_SECRET_ACCESS_KEY
```

### Extra steps for Windows
#### Load Credentials From Windows

Windows you have the option of installing [awscli for Windows](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) and then exposing your credentials in your Ubuntu container. This allows you to manage one set of credentials.
you have the option of installing [awscli for Windows](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) and then exposing your credentials in your WSL Linux environment. This allows you to manage one set of credentials for Windows and all Linux distros.

Assuming your username is `alex`, once you've installed awscli in Windows, confirm you now have the following files under Windows.

Expand Down
17 changes: 16 additions & 1 deletion website/docs/tutorials/newproject.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,12 @@ cd /workspaces
npx @seasketch/[email protected] init 7.0.0-beta.13
```

If you want to install anything other than the latest version you have to run a more specific command. Assume you want to init a new project using version 6.1.2. You would run:

```sh
npx @seasketch/[email protected] init 6.1.2
```

Tips:

- the answers to all of the init questions can be changed later, so don't worry if you don't know the answer.
Expand All @@ -95,6 +101,8 @@ Learn more about your projects [structure](../structure.md)
- [Low-level raster report](./sampleproject.md#seamount-report)
- [High-level raster report](./sampleproject.md#coral-species-report)

If you use the Metric Group feature, there are [examples](../concepts/Concepts.md) of variations for creating them based on the structure of your data.

## Build Your Project

The application `build` proceess packages it for deployment. Specifically it:
Expand Down Expand Up @@ -143,7 +151,14 @@ It will ask you if you want to publish all datasources, or choose from a list.

Your datasources will need to have already been imported using `import:data` and exist in the `data/dist` for this to work.

Note if you don't publish your datasources, then your local smoke tests may work properly, but your geoprocessing functions will throw file not found errors in production.
Note, if you don't publish your datasources, then your local smoke tests may work properly, but your geoprocessing functions will throw file not found errors in production.

Note, you can use any and all `aws` commands to look at your datasets bucket or to copy files to/from it.

```bash
aws s3 ls s3://gp-my-project-datasets
aws s3 cp data/dist/myFile.fgb s3://gp-my-project-datasets
```

## Integrate With SeaSketch

Expand Down
Loading

0 comments on commit dd1c40a

Please sign in to comment.