add profile for arcc

[rpetit3]

---

name: arcc
about: A profile for the ARCC HPC system at the University of Wyoming, USA

NOTE: This is my first submission to nf-core/configs. Any feedback or best practices will be most welcomed!

Please follow these steps before submitting your PR:

• If your PR is a work in progress, include [WIP] in its title
• Your PR targets the master branch
• You've included links to relevant issues, if any

Steps for adding a new config profile:

• Add your custom config file to the conf/ directory
• Add your documentation file to the docs/ directory
• Add your custom profile to the nfcore_custom.config file in the top-level directory
• Add your custom profile to the README.md file in the top-level directory
• Add your profile name to the profile: scope in .github/workflows/main.yml

[jfy133]

Overall good config and docs, but I'm not particularly happy with the custom parameters... I've left suggestions for alternative strategies

[jfy133]

I'm not sure about the use of these custom parameters. They may result in red herring warnings in both nextflow and nf-core pipelines as they aren't in the code itself/schema. Although it's probably solved by adding to the ignore parameters, you have the risk of someone specifies their own custom config to ignore other parameters, you would lose your ignore list.

We've recommended to others to ask users to specify bash env variables with the information, and pull these in the config with the groovy function I've forgotten the name of now... But I'm pretty sure @nvnieuwk and @matthdsm use in the VSC Ghent profile (?)

P.s. it's 5am and I'm on my phone thus the less than ideal referencing 😅

[rpetit3]

I've seen mixed usage here. I think I could probably get away with a SBATCH_ACCOUNT env variable to be consistent with other configs

[jfy133]

Rather than specifying this with custom parameters, why not simple sub profiles within this config, so: -profile arcc,teton

[rpetit3]

What happens then both partitions are to be used?

Should I create a teton_moran profile along with individual profiles?

[rpetit3]

here's an example: https://github.com/nf-core/configs/pull/622/files#diff-a45c821aa6dbf5cca764ee471f443619a08647e41cf112ef31b68831094ac821R178-R191

[rpetit3]

I wonder if it would be ok to just have a default that uses the three partitions

-profile arcc

Then use the additional profiles to explicitily set the partition

-profile arcc,beartooth_bigmem

Haha I'm trying to avoid

-partition arcc,beartooth_moran_teton

[rpetit3]

Sorry for spamming, but the reasoning for putting multiple partitions is to squeeze small jobs into available nodes with little wait time

[jfy133]

Yeah I think the profile example makes sense 👍

[jfy133]

What sort of things do you think would be going here? Isn't there a risk of a clash with existing opts that Slurm specifies by default?

[rpetit3]

Here would be additional GPU related options

[rpetit3]

@jfy133 thank you very much for the suggestions and feedback, I will make adjustments

[rpetit3]

@jfy133 I think I addressed most things

Any constructive feedback would be much appreciated! Haha like I said learning what best approach is here.

[jfy133]

I think that's OK, assuming it's been tested and working as intended 👍

[rpetit3]

OK! Thank you so much for all the feedback @jfy133

Did a bunch of testing. I tested each profile, found a few that had been retired but still listed in docs. I also verfied the .command.run files used the correct partitions.

nextflow run nf-core/fetchngs -profile test,arcc -c nfcore_custom.config --outdir test --skip_fastq_download
nextflow run nf-core/fetchngs -profile test,arcc,beartooth -c nfcore_custom.config --outdir test --skip_fastq_download
nextflow run nf-core/fetchngs -profile test,arcc,beartooth_bigmem -c nfcore_custom.config --outdir test --skip_fastq_download
nextflow run nf-core/fetchngs -profile test,arcc,beartooth_hugemem -c nfcore_custom.config --outdir test --skip_fastq_download
nextflow run nf-core/fetchngs -profile test,arcc,moran -c nfcore_custom.config --outdir test --skip_fastq_download
nextflow run nf-core/fetchngs -profile test,arcc,teton -c nfcore_custom.config --outdir test --skip_fastq_download
nextflow run nf-core/fetchngs -profile test,arcc,teton_cascade -c nfcore_custom.config --outdir test --skip_fastq_download
nextflow run nf-core/fetchngs -profile test,arcc,teton_hugemem -c nfcore_custom.config --outdir test --skip_fastq_download
nextflow run nf-core/fetchngs -profile test,arcc,teton_knl -c nfcore_custom.config --outdir test --skip_fastq_download

I think this will work out quite nicely, thank you very much for the profile suggestion!

[jfy133]

Go ahead and merge once you're ready!

[rpetit3]

Thank you @jfy133 !