Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linting and authorship info for CITATIONS #16

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
73 changes: 53 additions & 20 deletions CITATION.cff
Original file line number Diff line number Diff line change
@@ -1,56 +1,89 @@
# This CITATION.cff file was generated with cffinit.
# Visit https://bit.ly/cffinit to generate yours today!

---
cff-version: 1.2.0
title: HPC Workflow Management with Snakemake
title: "HPC Workflow Management with Snakemake"
message: >-
If you use this software, please cite it using the
metadata from this file.
type: software
authors:
- given-names: Alan
family-names: O'Cais
email: [email protected]
affiliation: University of Barcelona
orcid: 'https://orcid.org/0000-0002-8254-8752'
repository-code: 'https://github.com/carpentries-incubator/hpc-workflows'
url: 'https://carpentries-incubator.github.io/hpc-workflows/'
email: "[email protected]"
affiliation: "University of Barcelona"
orcid: "https://orcid.org/0000-0002-8254-8752"
alias: ocaisa
- given-names: Andrew
family-names: Reid
email: "[email protected]"
affiliation: "National Institute of Standards and Technology"
orcid: "https://orcid.org/0000-0002-1564-5640"
alias: reid-a
- given-names: Annajiat
family-names: Alim Rasel
email: "[email protected]"
affiliation: "Brac University"
orcid: "https://orcid.org/0000-0003-0198-3734"
alias: annajiat
- given-names: Benson
family-names: Muite
email: "[email protected]"
affiliation: "Kichakato Kizito"
alias: bkmgit
- given-names: Trevor
family-names: Keller
email: "[email protected]"
affiliation: "National Institute of Standards and Technology"
orcid: "https://orcid.org/0000-0002-2920-8302"
alias: tkphd
- given-names: Wirawan
family-names: Purwanto
email: "[email protected]"
affiliation: "Old Dominion University"
orcid: "https://orcid.org/0000-0002-2124-4552"
alias: wirawan0

repository-code: "https://github.com/carpentries-incubator/hpc-workflows"
url: "https://carpentries-incubator.github.io/hpc-workflows/"
abstract: >-
When using HPC resources, it's very common to need to
carry out the same set of tasks over a set of data
(commonly called a workflow or pipeline). In this lesson
we will make an experiment that takes an application which
runs in parallel and investigate it’s scalability. To do
runs in parallel and investigate its scalability. To do
that we will need to gather data, in this case that means
running the application multiple times with different
numbers of CPU cores and recording the execution time.
Once weve done that we need to create a visualisation of
Once we've done that we need to create a visualisation of
the data to see how it compares against the ideal case.


We could do all of this manually, but there are useful
tools to help us manage data analysis pipelines like we
have in our experiment. In the context of this lesson,
well learn about one of those: Snakemake.
we'll learn about one of those: Snakemake.
keywords:
- HPC
- Carpentries
- Lesson
- Workflow
- Pipeline
license: CC-BY-4.0
license: "CC-BY-4.0"
references:
- authors:
- title: "Getting Started with Snakemake"
authors:
- family-names: Collins
given-names: Daniel
title: "Getting Started with Snakemake"
alias: DC23
type: software
repository-code: 'https://github.com/carpentries-incubator/workflows-snakemake/'
url: 'https://carpentries-incubator.github.io/workflows-snakemake/'
- authors:
repository-code: "https://github.com/carpentries-incubator/workflows-snakemake"
url: "https://carpentries-incubator.github.io/workflows-snakemake/"
- title: "Snakemake for Bioinformatics"
authors:
- family-names: Booth
given-names: Tim
title: "Snakemake for Bioinformatics"
alias: tbooth
orcid: "https://orcid.org/0000-0003-2470-9519"
type: software
repository-code: 'https://github.com/carpentries-incubator/snakemake-novice-bioinformatics/'
url: 'https://carpentries-incubator.github.io/snakemake-novice-bioinformatics'
repository-code: "https://github.com/carpentries-incubator/snakemake-novice-bioinformatics/"
url: "https://carpentries-incubator.github.io/snakemake-novice-bioinformatics"
1 change: 0 additions & 1 deletion CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,5 @@ we pledge to follow the [The Carpentries Code of Conduct][coc].
Instances of abusive, harassing, or otherwise unacceptable behavior
may be reported by following our [reporting guidelines][coc-reporting].


[coc-reporting]: https://docs.carpentries.org/topic_folders/policies/incident-reporting.html
[coc]: https://docs.carpentries.org/topic_folders/policies/code-of-conduct.html
24 changes: 11 additions & 13 deletions config.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#------------------------------------------------------------
# Values for this lesson.
#------------------------------------------------------------
# HPC Workflow Management with Snakemake:
# Lesson templating & build parameters
---

# Which carpentry is this (swc, dc, lc, or cp)?
# swc: Software Carpentry
Expand All @@ -11,7 +11,7 @@
carpentry: 'incubator'

# Overall title for pages.
title: 'HPC Workflow Management with Snakemake' # FIXME
title: 'HPC Workflow Management with Snakemake'

# Date the lesson was created (YYYY-MM-DD, this is empty by default)
created: 2023-04-19
Expand All @@ -24,7 +24,7 @@ keywords: 'HPC Carpentry, snakemake, workflows, hpc'
life_cycle: 'pre-alpha'

# License of the lesson
license: 'CC-BY 4.0'
license: 'CC-BY-4.0' # SPDX identifier

# Link to the source repository for this lesson
source: 'https://github.com/carpentries-incubator/hpc-workflows'
Expand Down Expand Up @@ -59,12 +59,12 @@ contact: '[email protected]'

# Order of episodes in your lesson
episodes:
- 01-introduction.md
- 02-snakemake_on_the_cluster.md
- 03-placeholders.md
- 04-snakemake_and_mpi.md
- 05-chaining_rules.md
- 06-expansion.md
- 01-introduction.md
- 02-snakemake_on_the_cluster.md
- 03-placeholders.md
- 04-snakemake_and_mpi.md
- 05-chaining_rules.md
- 06-expansion.md

# Information for Learners
learners:
Expand All @@ -79,5 +79,3 @@ profiles:
#
# This space below is where custom yaml items (e.g. pinning
# sandpaper and varnish versions) should live


5 changes: 2 additions & 3 deletions environment.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
# Conda environment for the HPC Carpentry Workflows lesson
#
---
name: workflows
channels:
Expand All @@ -20,5 +19,5 @@ dependencies:
- radian # more colorful R REPL
- snakemake # workflow manager
- pip: # list of PyPI packages (not in Conda)
- amdahl # HPC Carpentry gray-box parallel program
- termplotlib # ASCII art plots
- amdahl # HPC Carpentry gray-box parallel program
- termplotlib # ASCII art plots
25 changes: 18 additions & 7 deletions episodes/01-introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,16 @@ exercises: 30
---

::: questions

- "How do I run a simple command with Snakemake?"

:::

:::objectives

- "Create a Snakemake recipe (a Snakefile)"
:::

:::

## What is the workflow I'm interested in?

Expand All @@ -37,6 +40,7 @@ which prints out the name of the host where the command is executed:
```bash
[ocaisa@node1 ~]$ hostname
```

```output
node1.int.jetstream2.hpc-carpentry.org
```
Expand Down Expand Up @@ -74,7 +78,7 @@ rule hostname_login:
1. We named the rule `hostname_login`. You may use letters, numbers or
underscores, but the rule name must begin with a letter and may not be a
keyword.
1. The keywords `input`, `output`, `shell` are all followed by a colon.
1. The keywords `input`, `output`, and `shell` are all followed by a colon (":").
1. The file names and the shell command are all in `"quotes"`.
1. The output filename is given before the input filename. In fact, Snakemake
doesn't care what order they appear in but we give the output first
Expand All @@ -85,10 +89,10 @@ rule hostname_login:
:::

Back in the shell we'll run our new rule. At this point, if there were any
missing quotes, bad indents, etc. we may see an error.
missing quotes, bad indents, etc., we may see an error.

```bash
$ snakemake -j1 -p hostname_login
snakemake -j1 -p hostname_login
```

::: callout
Expand All @@ -98,6 +102,7 @@ $ snakemake -j1 -p hostname_login
If your shell tells you that it cannot find the command `snakemake` then we need
to make the software available somehow. In our case, this means searching for
the module that we need to load:

```bash
module spider snakemake
```
Expand All @@ -122,7 +127,6 @@ Names marked by a trailing (E) are extensions provided by another module.

$ module spider snakemake/8.2.1
--------------------------------------------------------------------------------------------------------

```

Now we want the module, so let's load that to make the package available
Expand All @@ -136,9 +140,14 @@ and then make sure we have the `snakemake` command available
```bash
[ocaisa@node1 ~]$ which snakemake
```

```output
/cvmfs/software.eessi.io/host_injections/2023.06/software/linux/x86_64/amd/zen3/software/snakemake/8.2.1-foss-2023a/bin/snakemake
```

```bash
snakemake -j1 -p hostname_login
```
:::

::: challenge
Expand All @@ -152,8 +161,10 @@ What does the `-p` option in the `snakemake` command above do?
1. Tells Snakemake to only run one process at a time
1. Prompts the user for the correct input file

*Hint: you can search in the text by pressing `/`, and quit back to the shell
with `q`*
:::::: hint
You can search in the text by pressing <kbd>/</kbd>,
and quit back to the shell with <kbd>q</kbd>.
::::::

:::::: solution
(2) Prints the shell commands that are being run to the terminal
Expand Down
Loading
Loading