Skip to content

Commit

Permalink
Merge pull request #27 from ACED-IDP/feature/sdk-query
Browse files Browse the repository at this point in the history
`feature/sdk-query` → `main`: Add steps for using the Gen3 SDK to query Metadata/Guppy
  • Loading branch information
lbeckman314 authored Dec 14, 2024
2 parents c3b4f33 + 3c220fd commit 8552e70
Show file tree
Hide file tree
Showing 6 changed files with 124 additions and 6 deletions.
File renamed without changes
File renamed without changes
4 changes: 2 additions & 2 deletions docs/requirements/gen3-client.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,11 +89,11 @@ In such a case please reach out to the ACED development team for assistance.

Before using the gen3-client to upload or download data, the `gen3-client` needs to be configured with API credentials downloaded from the [Profile page](https://aced-idp.org/identity).

![Gen3 Profile page](profile.png)
![Gen3 Profile page](/images/api-key.png)

Download the access key from the portal and save it in the standard location `~/.gen3/credentials.json`

![Gen3 Credentials](credentials.png)
![Gen3 Credentials](/images/credentials.png)

From the command-line, run the gen3-client configure command:

Expand Down
112 changes: 112 additions & 0 deletions docs/workflows/query.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@

# Query

## Overview ⚙️

Gen3 supports API access to Files and Metadata, allowing users to download and query their data via the Gen3 SDK and GraphQL queries.

## Quick Start ⚡️

*Adapted from the [Gen3 SDK Quick Start page](https://github.com/uc-cdis/gen3sdk-python/blob/master/docs/tutorial/quickStart.md)*

## 1. Install

The Gen3 SDK is available for installation via PyPi:

```sh
pip install gen3
```

## 2. Authenticate

??? "Configure a Profile with Credentials"

Download an API Key from the [Profile page](https://aced-idp.org/identity) and save it to `~/.gen3/credentials.json`

![Gen3 Profile page](/images/api-key.png)

![Gen3 Credentials](/images/credentials.png)

## 3. Query

### Query (`example.graphql`)
```js
query ExampleQuery {
files: file(first: 1000) {
file_name
project_id
id
}
patients: patient(first: 1000) {
name
project_id
id
}
observations: observation(first: 1000) {
code
project_id
id
}
}
```

### Script (`example.py`)
```sh
from gen3.auth import Gen3Auth
from gen3.query import Gen3Query
import json

auth = Gen3Auth()

query = ''

# Read in Example Query
with open('example.graphql') as f:
query = f.read()

response = Gen3Query.graphql_query(Gen3Query(auth), query_string=query)

formatted = json.dumps(response, indent=2)

print(formatted)

# >>> Example Output
```

### Output

```sh
$ python example.py
{
"data": {
"files": [
{
"file_name": "example.bam",
"project_id": "cbds-example",
},...
],
"patients": [
{
"name": "Example Name",
"project_id": "cbds-example",
},...
],
"observations": [
{
"code": "Example Code",
"project_id": "cbds-example",
},...
]
}
}
```

## Additional Resources 📚

- [Gen3 SDK Documentation](https://uc-cdis.github.io/gen3sdk-python/_build/html/index.html)

- [Gen3 SDK Repo](https://github.com/uc-cdis/gen3sdk-python)

- [Gen3 SDK PyPi](https://pypi.org/project/gen3)

- [Guppy Syntax Docs](https://github.com/uc-cdis/guppy/blob/master/doc/queries.md)
13 changes: 9 additions & 4 deletions docs/workflows/upload.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,10 +60,15 @@ Note that we use `xargs` `-P 0` argument to run commands in parallel, greatly re
The general form of adding remote file(s) is the following:
```sh
g3t add s3://<Bucket>/<File> \
--etag <ETag> \
--size <Size> \
--modified <Date> \
--no-git-add
--etag {ETag} \
--mime {mime_type} \
--modified {system_time} \
--no-git-add \
--patient {patient_id} \
--realpath {full_path} \
--size {system_size} \
--specimen {specimen_id} \
{static_parameters}
```

### Example
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ nav:
- Exploring Data:
- workflows/portal-explore.md
- workflows/portal-download.md
- workflows/query.md
- Utilities:
- workflows/approve-requests.md
- workflows/add-users.md
Expand Down

0 comments on commit 8552e70

Please sign in to comment.