Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: Migrate to pyproject.toml #241

Merged
merged 34 commits into from
Jan 13, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
106d67a
move to src/ layout, update biomedsheets and sodar-cli deps
tedil Jan 10, 2025
44f78e6
more fixes
tedil Jan 10, 2025
5b4d5c2
switch to ruff, update github ci
tedil Jan 10, 2025
1341592
ruff fmt
tedil Jan 10, 2025
e2a58ba
fix docstring
tedil Jan 10, 2025
266433b
fix sphinxproj name
tedil Jan 10, 2025
30bd7b5
fix lints
tedil Jan 10, 2025
b38c09b
address more lints
tedil Jan 10, 2025
389f787
update release workflow
tedil Jan 10, 2025
e5412d5
define default shell for main gh workflow
tedil Jan 10, 2025
b15b998
remove pysam from dependencies (unused)
tedil Jan 10, 2025
c951cd8
also remove vcfpy
tedil Jan 10, 2025
e81971a
re-add vcfpy, which is indeed used
tedil Jan 10, 2025
60972dc
install vcfpy and pysam from env.yaml
tedil Jan 10, 2025
009d047
fix python version
tedil Jan 10, 2025
be02b7d
add libs needed for building pysam
tedil Jan 10, 2025
19305b1
add strategy
tedil Jan 10, 2025
c790508
replace deprecated logger.warn with logger.warning
tedil Jan 11, 2025
569d33f
symlink loop raises an OSError
tedil Jan 11, 2025
3835425
fix sodar_cli imports
tedil Jan 11, 2025
1ba8a43
use fs fixture in test_run_seasnap_itransfer_results_smoke_test
tedil Jan 13, 2025
b5d515f
bump python to 3.12, fix snappy and altamisa deps
tedil Jan 13, 2025
934fff9
include zlib in conda env
tedil Jan 13, 2025
01fbb1b
do not pull from lfs when installing via pip
tedil Jan 13, 2025
49b53ac
use GITHUB_ENV instead
tedil Jan 13, 2025
a170707
explicit strict=True in zip
tedil Jan 13, 2025
98bef5e
use fs fixture
tedil Jan 13, 2025
34945e6
update installation instructions
tedil Jan 13, 2025
ef2ecd9
add information about snappy optional dep
tedil Jan 13, 2025
cb37c3c
fix typo
tedil Jan 13, 2025
fc36c72
skip test_run_archive_readme_smoke_test for now
tedil Jan 13, 2025
3347b2f
update .readthedocs configuration
tedil Jan 13, 2025
39143b4
update: how to build docs
tedil Jan 13, 2025
0c70bfa
update README for installing snappy extra
tedil Jan 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 46 additions & 35 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,10 @@ on:
pull_request:
branches_ignore: []

defaults:
run:
shell: bash -el {0}

jobs:
cancel-previous:
runs-on: ubuntu-latest
Expand All @@ -20,18 +24,34 @@ jobs:

linting:
runs-on: ubuntu-latest
strategy:
matrix:
python-version:
- "3.12"
steps:
- uses: actions/checkout@v1
- name: Checkout repository
uses: actions/checkout@v4
with:
lfs: true

- name: Install system dependencies
run: sudo apt-get install -y libbz2-dev liblzma-dev

- name: Setup environment variables
run: echo "GIT_LFS_SKIP_SMUDGE=1" >> $GITHUB_ENV

- name: Install Python
uses: actions/setup-python@v2
- name: Setup conda
uses: conda-incubator/setup-miniconda@v3
with:
python-version: "3.10"
channels: conda-forge,bioconda,nodefaults
environment-file: 'environment.yaml'
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: |
pip install -r requirements/test_black.txt
pip freeze
- name: Set up Python
run: uv python install ${{ matrix.python-version }}

- name: Install the project
run: uv sync --all-extras --dev

- name: Run linting tools
run: |
Expand All @@ -41,51 +61,42 @@ jobs:
if: github.event_name == 'pull_request' && failure()
uses: marocchino/[email protected]
with:
message: 'Please format your code with [black](https://black.readthedocs.io): `make black`.'
message: 'Please check your code via `make lint`.'
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

testing:
runs-on: ubuntu-latest
strategy:
matrix:
python-version:
- '3.10'
# - "3.11" # no compatible pysam yet
# - "3.12" # no compatible pysam yet
- "3.12"
needs: linting
steps:
- name: Install Python via conda
uses: s-weigand/setup-conda@v1
with:
python-version: ${{ matrix.python-version }}
conda-channels: defaults,bioconda,conda-forge

- name: Checkout repository
uses: actions/checkout@v2
uses: actions/checkout@v4
with:
lfs: true

- name: Install mamba
run: conda install -y mamba
- name: Install system dependencies
run: sudo apt-get install -y libbz2-dev liblzma-dev

- name: Prepare environment.yaml file
run: >
cp environment.yaml /tmp/environment.yaml && sed -i -e
's/- python/- python=${{ matrix.python-version }}/'
/tmp/environment.yaml
- name: Setup environment variables
run: echo "GIT_LFS_SKIP_SMUDGE=1" >> $GITHUB_ENV

- name: Update environment using mamba
run: mamba env update --name root --file /tmp/environment.yaml
- name: Setup conda
uses: conda-incubator/setup-miniconda@v3
with:
channels: conda-forge,bioconda,nodefaults
environment-file: 'environment.yaml'
python-version: ${{ matrix.python-version }}

- name: Save time by installing packages via mamba.
run: mamba install -y pysam
- name: Set up Python
run: uv python install ${{ matrix.python-version }}

- name: Install dependencies
run: |
pip install -r requirements/test_black.txt
pip freeze
- name: Install the project
run: uv sync --all-extras --dev

- name: Run tests
run: pytest
run: uv run pytest

- uses: codecov/codecov-action@v3
15 changes: 7 additions & 8 deletions .github/workflows/release-please.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,28 +10,27 @@ jobs:
runs-on: ubuntu-latest
steps:

- uses: GoogleCloudPlatform/release-please-action@v2
- uses: GoogleCloudPlatform/release-please-action@v4
id: release
with:
release-type: python
package-name: cubi-tk

- uses: actions/checkout@v2
- uses: actions/checkout@v4
if: ${{ steps.release.outputs.release_created }}
with:
fetch-depth: 0

- name: Set up Python
- name: Setup uv
if: ${{ steps.release.outputs.release_created }}
uses: actions/setup-python@v2
uses: astral-sh/setup-uv@v3
with:
python-version: '3.x'
version: "0.5.16"
enable-cache: true

- name: Build package
if: ${{ steps.release.outputs.release_created }}
run: |
python -m pip install --upgrade pip
python setup.py sdist
uv build

- name: Publish to PyPI
if: ${{ steps.release.outputs.release_created }}
Expand Down
25 changes: 25 additions & 0 deletions .readthedocs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# .readthedocs.yaml
# Read the Docs configuration file
# See https://docs.readthedocs.io/en/stable/config-file/v2.html for details

# Required
version: 2

# Set the version of Python and other tools you might need
build:
os: ubuntu-22.04
jobs:
post_create_environment:
- pip install uv
- VIRTUAL_ENV=$READTHEDOCS_VIRTUALENV_PATH uv pip install --upgrade pip setuptools
- VIRTUAL_ENV=$READTHEDOCS_VIRTUALENV_PATH uv pip install --upgrade sphinx readthedocs-sphinx-ext
post_install:
- VIRTUAL_ENV=$READTHEDOCS_VIRTUALENV_PATH uv sync --all-extras --group docs
- VIRTUAL_ENV=$READTHEDOCS_VIRTUALENV_PATH uv pip install '.[all]'
build:
html:
- VIRTUAL_ENV=$READTHEDOCS_VIRTUALENV_PATH uv run sphinx-build --fail-on-warning --show-traceback --keep-going -D language=de docs_manual $READTHEDOCS_OUTPUT/html

## Build documentation in the docs/ directory with Sphinx
#sphinx:
# configuration: docs_manual/conf.py
4 changes: 2 additions & 2 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@ include CHANGELOG.md
include LICENSE
include README.md

graft src

recursive-include tests *
recursive-exclude * __pycache__
recursive-exclude * *.py[co]

recursive-include docs *.rst conf.py Makefile make.bat *.jpg *.png *.gif

include cubi_tk/version.py
recursive-include cubi_tk/isa_tpl *.json *.txt

include requirements.txt requirements/*.txt
37 changes: 10 additions & 27 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,35 +1,18 @@
.PHONY: all
all: black
all: lint format

.PHONY: black
black:
black -l 100 .
.PHONY: format
format:
uv run ruff format .

.PHONY: black-check
black-check:
black -l 100 --check .

.PHONY: isort
isort:
isort --force-sort-within-sections --profile=black .

.PHONY: isort-check
isort-check:
isort --force-sort-within-sections --profile=black --check .

.PHONY: flake8
flake8:
flake8

# .PHONY: mypy
# mypy: export MYPYPATH=stubs
# mypy:
# mypy clinvar_this clinvar_api tests
.PHONY: format-check
format-check:
uv run ruff format --check .

.PHONY: lint
lint: flake8 isort-check black-check
# lint: flake8 isort-check black-check mypy
lint:
uv run ruff check

.PHONY: pytest
pytest:
pytest .
uv run pytest
11 changes: 7 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Tooling for connecting GitLab, pipelines, and SODAR at CUBI.
Prerequisites when using conda:

```bash
$ conda create -n cubi-tk python=3.10
$ conda env create -n cubi-tk -f environment.yaml
$ conda activate cubi-tk
```

Expand All @@ -21,15 +21,18 @@ Clone CUBI-TK and install.
```bash
$ git clone [email protected]:bihealth/cubi-tk.git
$ cd cubi-tk
$ pip install -e .
$ uv sync
$ uv pip install -e .
# or, if you need snappy kickoff:
#$ GIT_LFS_SKIP_SMUDGE=1 uv pip install -e '.[snappy]'
```

## Building the Manual

```bash
$ pip install -r requirements/develop.txt
$ uv sync --all-extras --group docs
$ cd docs_manual
$ make clean html
$ uv run make clean html
$ ls _build/html/index.html
```

Expand Down
1 change: 0 additions & 1 deletion cubi_tk/__init__.py

This file was deleted.

2 changes: 1 addition & 1 deletion docs_manual/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = python -msphinx
SPHINXPROJ = altamisa
SPHINXPROJ = cubi-tk
SOURCEDIR = .
BUILDDIR = _build

Expand Down
2 changes: 1 addition & 1 deletion docs_manual/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
language = "en"

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
Expand Down
2 changes: 1 addition & 1 deletion docs_manual/make.bat
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ if "%SPHINXBUILD%" == "" (
)
set SOURCEDIR=.
set BUILDDIR=_build
set SPHINXPROJ=altamisa
set SPHINXPROJ=cubi-tk

if "%1" == "" goto help

Expand Down
9 changes: 7 additions & 2 deletions environment.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,16 @@
channels:
- conda-forge
- bioconda
- defaults
- nodefaults

name: cubi-tk

dependencies:
- python
- python >=3.9
- pip
- uv >=0.5
- hashdeep
- pysam >=0.22
- vcfpy >=0.13.8
- gcc_linux-64 >=13,<14
- zlib ~=1.3.0
Loading
Loading