Skip to content

Commit

Permalink
added +1000 instead of 999 to dev-build-numbers
Browse files Browse the repository at this point in the history
  • Loading branch information
lukas.gehrig committed Nov 30, 2023
1 parent 74fd195 commit 120d3b7
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 3 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/build_base.yml
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ jobs:
BUILD_NR=${{ inputs.buildNumber }}
if [[ ${{ inputs.dev }} == "true" ]]
then
BUILD_NR="$(($BUILD_NR + 999))"
BUILD_NR="$(($BUILD_NR + 1000))"
PKG_VERSION=$(echo "${{ inputs.packageVersion }}" | sed 's/\([0-9]*\.[0-9]*\.\)\([0-9]*\)/\1dev\2/g')
DEV_DEPS_LIST=$(echo "${{ inputs.packageVersionName }}, ${{ inputs.devDeps }}")
else
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,11 @@ Dev-builds can have other conda-dependencies which are also part of the Feedstoc

**Constrain Dependencies**: We want to be nice to users of Karabo so they are able to install additional packages without beeing at risk to break an environment. To enable that, we should be sure to not constrain well-known dependencies like `numpy`, `scipy` or `astropy` too much if possible. On the other hand, be sure to constrain packages with a lot of api-breaking changes and exclude versions of well-known packages with known issues (for Karabo). This approach relies on a lot of testing. But since we build a lot of conda-wheels of 3th party packages, it's too cumbersome to test them individually. Therefore, the aim should be to have a high test-coverage of Karabo.

**About Build-Strings and Versions**: The uniqueness of a conda-wheel is determined by the uniqueness of the build-string and NOTHING ELSE. So if another wheel has the exact same build-string, even if it's build completely different, it would overwrite an already existing conda-wheel on [anaconda.org](https://anaconda.org/). This can brake package installations which depend on the accidentally removed package. So if you create a new `meta.yaml`, pay special attention to the `build` section to prevent issues. When creating a new conda build, please increase the `build-number` for each new build of the same package version, so conda knows which version to install. [NON-MPI-based](#about-mpi) build-numbers have an offset of +100 if and only if there are different mpi and non-mpi wheels of the same package version on anaconda.org. This let's anaconda prefer nompi-wheels before mpi-wheels. Keep that in mind by triggering OR building mpi-builds. So if there is e.g. an 102 mpich build on anaconda, for the next build of the same version, the input must be `3` and NOT `103` because the `meta.yaml` should take care of the increasing offset. Build-numbers of [dev-builds](#dev-builds) have an offset of +999 to definitely not overwrite an existing wheel even if the build-recipe is exactly the same.
**About Build-Strings and Versions**: The uniqueness of a conda-wheel is determined by the uniqueness of the build-string and NOTHING ELSE. So if another wheel has the exact same build-string, even if it's build completely different, it would overwrite an already existing conda-wheel on [anaconda.org](https://anaconda.org/). This can brake package installations which depend on the accidentally removed package. So if you create a new `meta.yaml`, pay special attention to the `build` section to prevent issues. When creating a new conda build, please increase the `build-number` for each new build of the same package version, so conda knows which version to install. [NON-MPI-based](#about-mpi) build-numbers have an offset of +100 if and only if there are different mpi and non-mpi wheels of the same package version on anaconda.org. This let's anaconda prefer nompi-wheels before mpi-wheels. Keep that in mind by triggering OR building mpi-builds. So if there is e.g. an 102 mpich build on anaconda, for the next build of the same version, the input must be `3` and NOT `103` because the `meta.yaml` should take care of the increasing offset. Build-numbers of [dev-builds](#dev-builds) have an offset of +1000 to definitely not overwrite an existing wheel even if the build-recipe is exactly the same.

A normal build-string for a python package (according to [conda-variants doc](https://conda.io/projects/conda-build/en/latest/resources/variants.html#differentiating-packages-built-with-different-variants)) looks as follows: `py{{ CONDA_PY }}h{{ PKG_HASH }}_{{ BUILD_NUMBER }}`. We usually pin a package to a specific python version (even though it may be compatible with other python versions) to ensure compatibility of a package with a specific python version. The python version of each package is pinned in each `conda_build_config.yaml`. In case we want to support a range of python versions, where the package dependencies differ, you can define the constraints in `conda_build_config.yaml` and [zip](https://docs.conda.io/projects/conda-build/en/stable/resources/variants.html#special-variant-keys) them with the python version.

Errors in the build-string can occur if they have an `-` (see [Build number and string](https://docs.conda.io/projects/conda-build/en/stable/resources/define-metadata.html#build-number-and-string)). **Attention**: For some reason, the build-string of the according `meta.yaml` gets ignored in some builds and therefore the package-hash is missing in the build-string. Not sure why this is happening. There were cases where the [Host](https://docs.conda.io/projects/conda-build/en/stable/resources/define-metadata.html#host) and/or the [Build](https://docs.conda.io/projects/conda-build/en/stable/resources/define-metadata.html#build) section of the `meta.yaml` were corrupted (e.g. too many unnecessary tools installed) and then the build-string was not build like specified in the according `meta.yaml`. This can have a huge impact if you don't know what you're doing if you trigger a new build. If you don't set the build-number properly, it can easily overwrite an existing live-build on [anaconda.org](https://anaconda.org/i4Ds). This is because if a package is overwritten or not solely relies on the uniqueness of the build-string, and without the hash it just consists of the package-version and the build-number. However, if you use a dev-build, you don't have to worry (make sure that the flag is set!!!), since they have an offset of the build-number of +999).
Errors in the build-string can occur if they have an `-` (see [Build number and string](https://docs.conda.io/projects/conda-build/en/stable/resources/define-metadata.html#build-number-and-string)). **Attention**: For some reason, the build-string of the according `meta.yaml` gets ignored in some builds and therefore the package-hash is missing in the build-string. Not sure why this is happening. There were cases where the [Host](https://docs.conda.io/projects/conda-build/en/stable/resources/define-metadata.html#host) and/or the [Build](https://docs.conda.io/projects/conda-build/en/stable/resources/define-metadata.html#build) section of the `meta.yaml` were corrupted (e.g. too many unnecessary tools installed) and then the build-string was not build like specified in the according `meta.yaml`. This can have a huge impact if you don't know what you're doing if you trigger a new build. If you don't set the build-number properly, it can easily overwrite an existing live-build on [anaconda.org](https://anaconda.org/i4Ds). This is because if a package is overwritten or not solely relies on the uniqueness of the build-string, and without the hash it just consists of the package-version and the build-number. However, if you use a dev-build, you don't have to worry (make sure that the flag is set!!!), since they have an offset of the build-number of +1000).

## About MPI

Expand Down

0 comments on commit 120d3b7

Please sign in to comment.