Skip to content

Release Process

Jump to bottom
Erik Jaegervall (Jägervall) edited this page Dec 11, 2024 · 8 revisions

Make sure we are feature complete

No remaining important PRs or issues?

Check links

The commands below can be used to verify that links in markdown files works as expected

npm install -g markdown-link-check

find . -name \*.md -print0 | xargs -0 -n1 markdown-link-check

Or stricter:

markdown-link-check -q -a 403,200,404 *.md

To check all but ignore third party (submodule) code do from root repo folder:

find . -name \*.md -print0 | xargs -0 -n1 markdown-link-check -q -a 403,200,404

Update Dependencies

For all releases except patch releases we intend to update dependencies to latest. (Unless there is a good reason not to do it). The reason for this is that it is more likely that security vulnerabilities (if detected) will be fixed on recent releases. If you are a committer, check that no vulnerabilities are reported in the Security tab

First step is to update versions in pyproject.toml and setup.cfg if needed. For grpcio-tools it is important to use the same version in pyproject.toml (which controls which version to use when generating grpc files) and setup.cfg (which indirectly through generated requirements.txt controls which version to use in runtime.)

Then update exact versions

cd kuksa-client
pip-compile --upgrade --resolver=backtracking setup.cfg
pip-compile --upgrade --extra=test --output-file=test-requirements.txt --resolver=backtracking setup.cfg

Create Pull Request

If you changed something, create a pull request, have it reviewed and merged and verify that all checks pass

Creating Releases/Pre-releases for kuksa-client

Do not use tags of the form 1.2.3a1 for pre-releases, instead use 1.2.3-a1

The repo use semantic versioning for both docker images and pypi packages. Pep 0440 describes Python versioning scheme. It can be noted that semver and Pep have somewhat different requirements.

If you have created a pypi release then that number can only be reused if a kuksa-client pypi contributor actually deletes it. That is cumbersome.

  • If you are to create a final release, use a tag of the form X.Y.X, for example 0.5.0
  • If you are to create a pre-release for testing, use a tag of the form X.Y.Z-*. The dash is needed for ghcr, and will be ignored for pypi builds.
    • Use "alpha" for arbitrary builds during development, using intended next version as tag. If last release for example was 0.5.0, then a good pre-release is 0.5.1-a1 or 0.6.0-a1. If you need more increment the suffix, like a2, a3, and so on.
    • Use "rc" for release candidates, using intended next version as tag. If next intended release is 0.6.0, then a good tag is 0.6.0-rc1.

Get the commit you want from the remote, tag it and push the tag to the official kuksa-python-sdk remote.

Check build results, upon success there should be:

Manual build and upload to pypi

Note: Normally not needed as handled automatically when you push a tag like described above!

user@debian:~/kuksa.val/kuksa-client$ rm -rf dist 
user@debian:~/kuksa.val/kuksa-client$ python -m build
tag_filter is not a valid function reference: Wrong reference name: [0-9]+.[0-9]+.[0-9]+
* Creating virtualenv isolated environment...
* Installing packages in isolated environment... (grpcio-tools>=1.46.0, setuptools-git-versioning, setuptools>=42, wheel)
* Getting build dependencies for sdist...
tag_filter is not a valid function reference: Wrong reference name: [0-9]+.[0-9]+.[0-9]+
Matched 0.4.0a1
Matched 0.3.1
Matched 0.3.0

< lot of lines removed >

removing build/bdist.linux-x86_64/wheel
Successfully built kuksa_client-0.4.0a1.tar.gz and kuksa_client-0.4.0a1-py3-none-any.whl

The result can be uploaded to pypi or to the pypi test repository, if you have the rights to do so. Uploading to testpypi may have limited values for released packages.

python -m twine upload --repository testpypi dist/*
python -m twine upload --repository pypi dist/*

To be able to upload you need a config file, similar to this one:

user@debian:~/kuksa.val/kuksa-client$ more ~/.pypirc
[distutils]
  index-servers =
    pypi
    testpypi

[testpypi]
  username = __token__
  password = pypi-<token content>
  
  
[pypi]
  username = __token__
  password = pypi-<token content>

If successful it will be shown as a pre-release at pypi

image

Using pre-releases

By default pip does not install pre-releases, so you need to specify that with --pre. Just requesting kuksa-client to be installed will result in the latest stable release will be used:

user@debian:~/kuksa.val/kuksa-client$ pip install kuksa-client
Defaulting to user installation because normal site-packages is not writeable
Collecting kuksa-client
...
Successfully installed kuksa-client-0.3.1
user@debian:~/kuksa.val/kuksa-client$ pip install --pre kuksa-client
Defaulting to user installation because normal site-packages is not writeable
Collecting kuksa-client
...
Successfully installed kuksa-client-0.4.0a1

pip supports specifying multiple repositories, but if a package name exists in both only the official pypi repository will be checked.

# This will only check for kuksa-client in https://pypi.org/simple/
pip3 install --pre --index-url https://test.pypi.org/simple/  --extra-index-url https://pypi.org/simple/ kuksa-client==0.4.0a1

# This will use kuksa-client from test.pypi
pip3 install --pre --index-url https://test.pypi.org/simple/  kuksa-client==0.4.0a1

Using pre-releases in Kuksa Providers

To be able to use a pre-release two things must be modifed. The first version dependency like in requirements.in. There the dependency must be changed so it includes or possibly even require a newer version. For the 0.4.0a1 example the version requirement kuksa-client ~= 0.3.0 must be changed, for example to kuksa-client >= 0.4.0a1. That will accept any 0.4.* release (we hope), including pre-releases. Specifying kuksa-client ~= 0.4.0 does not work.

The requirements.txt file must be regenerated considering pre-releases.

pip-compile --upgrade --pre requirements.in

Finally any file using pip install must be updated to also include pre-releases, like Dockerfile

RUN /opt/venv/bin/python3 -m pip install --upgrade pip \
    && pip3 install --pre --no-cache-dir -r requirements.txt

Creating alpha PyPI packages is only needed if you cannot test with a locally installed kuksa-client package, for instance to build docker containers!

Publish Release

Releases created by CI are visible on the release page for committers, but they are not published. To publish do this:

  • Manually select previous release and generate release notes
  • Copy instructions based on previous release (part above ## What's Changed), i.e. information on how to fetch from docker
  • Also say something about the included *.zip files
  • Select "Set as a pre-release"
  • Publish

Then inform the Kuksa Project lead, they are the only one allowed to remove the "pre-release" label

Clean-up policy

  • Everything related to pre-releases may be deleted (alpha/rc tags and related content)
  • Real tags/releases (for version like 1.2.3) shall never be removed, neither shall branches handling that tag
Clone this wiki locally