Firedrake versioned releases

[dham]

This is a discussion about how we plan to move to having versioned releases of Firedrake. Right now this is a proposal and we need to work through some details.

Rationale

Most users don't need the latest features as soon as they land, but are susceptible to API changes and upstream breakages. Having changes happen at fixed intervals is more friendly to them. By pinning the upstream dependencies in a given release, we can hope to avoid unscheduled downstream breakages.

The risk is that users stick to an old version of Firedrake and then encounter breakages when uncontrolled upstream dependencies change (e.g. compilers and system libraries changing such as to break PETSc). This risk exists already since many users do not run Firedrake-update. We should mitigate the risk by being clear about what versions are supported.

PETSc release cycle

PETSc have a point release at the end of March and September and monthly patch releases between. See the comments to: https://gitlab.com/petsc/petsc/-/merge_requests/8090

Proposed Firedrake release cycle

Main Firedrake releases April and October, fixing where possible on the PETSc release of the previous month. If this introduces a breakage that we can't immediately fix, we can release with the previous PETSc release. This timing means we pick up PETSc changes as soon as we stably can.

Firedrake patch releases between main releases when we deem a change to be sufficiently immediately necessary to downstream users that the work involved in a patch release is justified.

Proposed version numbers

Given that we primarily propose to do calendar (as opposed to semantic) releases, it is proposed that version numbers are also primarily calendar-based. The form would be year.month.patch so the biannual releases for 2025 would be 2025.4.0 and 2025.10.0. Any patch releases would increase the patch number but not the month number. I.e. if the first patch to 2025.4.0 is released in June then it would have number 2025.4.1 and not 2025.6.0.

Unresolved issues

We need to work out how to be on stable releases of loopy and friends.

[connorjward]

How do we make sure that users only install the "blessed" version of PETSc that we claim to support? We can specify the petsc4py version in the pyproject.toml but for PETSc itself we need the user to check out the right git tag.

I have looked and I am fairly sure that PETSc configure does not allow us to specify this.

[dham]

Yeah. PETSc configure is too late. Maybe the firedrake-configure script has to do this. The workflow would then be:

curl firedrake-configure at the appropriate release tag.
git clone $(./firedrake-configure --petsc-url)
cd petsc
./configure $(../firedrake-configure --petsc-configure-options)
make

[connorjward]

This makes sense. I also didn't think to combine the different scripts into one curl-able thing.

[JHopeCollins]

Does petsc4py follow the same version numbering? Would it be possible to get pip install firedrake to complain if the wrong PETSc is used?

[connorjward]

Yes that's perfectly possible. We can also inspect PETSc directly for this information in one of its config files inside PETSC_DIR (which has to be set).

[dham]

I think it's even more straightforward. Firedrake's pyproject.toml should simply require the right version of petsc4py (i.e. the version that matches the petsc version that ./firedrake-configure --petsc-url returns.

[francesco-ballarin]

A clarification: will firedrake master branch still (try as best as possible to) be compatible with the latest PETSc master branch (i.e., not released yet)?
I do understand that firedrake 2025.4.0 will be compatible with PETSc 2025.3.0 (or whatever v3.2X in their versioning scheme). The question more or less can be equivalently reformulated as: say there is a breaking change in PETSc 2025.9.0 (or whatever v3.(2X + 1)), will this ideally get caught by firedrake between April and September 2025, or will catching this be delayed between September and October 2025?

[connorjward]

I feel like we would also want to be able to run our release branch against whatever PETSc release we claim to support. Otherwise making patches will be difficult. This would be straightforward in the CI.

[dham]

Think this thread is only about master. I think that the plan is that release is pinned (until we start preparing for the next release).

[connorjward]

But PETSc might make bugfixes to their release branch (e.g. if macOS releases an update), so we would want to be able to update which PETSc our release branch is for.

But I also think that tracking their main is a good idea.

[dham]

I think that we wouldn't automatically track their release branch. If there is an issue in petsc which they merge to release, I think we have to wait for their monthly point release and then update our release branch to point to that, and do a point release ourselves.

Otherwise we have our release branch tracking a moving target (albeit a less aggressively moving one than PETSc main) which is the one thing that we want our releases not to do.

[connorjward]

I agree.

[connorjward]

One thing which had me concerned for a while was the question of pinning our dependencies. Fairly frequently we encounter issues where an upstream package releases a new version that breaks things for us. I think there are two ways around this:

1. Set a maximum version for of all of our dependencies so we won't ever install anything newer.
2. When a breakage is observed make a patch release (to release branch) that constrains the package version (package<the_new_version). At the same time fix the main branch and set the version constraint to package>=the_new_version.

I think that 1 will lead to a lot of bit-rotting where we unnecessarily pin things and fall behind. 2 makes a lot more sense to me.

[dham]

Do we not want to pin at least known offenders in the release branch (but not in main). This would insulate users of release from the nasty surprises while future release versions update every 6 months.

[connorjward]

I guess that would be fine. Just as long as we don't have maximum version constraints in main.

[colinjcotter]

Would there be a single “release” branch that master merged into, or a new branch for each release?

[dham]

I think you have a new one every six months. So when we prepare for release, we make a release_new branch by forking master. Once we are ready, we tag current release as release_old_version_number and then force update release to release_new.

[connorjward]

I think we could just use a single release branch. You can then make releases by tagging commits.

[jrmaddison]

Would every release also be published on Zenodo? And firedrake-zenodo be updated to do the right thing if running from a release?