Upgrade Beiwe Backend platforms to Python 3.12

[biblicabeebli]

Current Status: Done, except for the Docker configuration.:

• Documentation has been added to the wiki at https://github.com/onnela-lab/beiwe-backend/wiki/Upgrading-Elastic-Beanstalk-from-Python-3.8-to-3.12
• We need to update on the docker containers.
• The changes are not yet pushed to the main branch for reasons not worth explaining here It's on main.
• We have yet to track down all references to "3.8." Other documentation has been updated

So far there are no runtime issues, it was all deployment infrastructure that needed updating.

---

Original post:

Python 3.8 goes out of support in October (2024-10, https://devguide.python.org/versions/ ).

We are currently blocked on upgrading to the most recent Python 3.12 due to the platform not being available on Elastic Beanstalk.

• [Package Request] - Python 3.12 amazonlinux/amazon-linux-2023#483
• Keep Python up-to-date aws/elastic-beanstalk-roadmap#174
• official reference https://docs.aws.amazon.com/elasticbeanstalk/latest/platforms/platform-history-python.html

Python platform upgrades are a somewhat complex to implement upgrade because we need to test and throw together directions like we have for the 3.6 to 3.8 upgrade, like we did here.

• https://github.com/onnela-lab/beiwe-backend/wiki/Upgrading-to-Python-3.8

Usually the process is a fairly simple sequence of commands usung the AWS and Elastic Beanstalk command line tools.

The real complexity is:

• we have to correctly document and mark a branch (see the bottom of the full branches list https://github.com/onnela-lab/beiwe-backend/branches/all for examples) so that there is a still-3.8-compatible branch a slow-to-upgrade platform sysadmin can reference and swap over to when they get around to upgrading.
• Development also requires testing updates to the Beiwe deployment tool launch script - a full stack deploy, which is ... annoying.

[biblicabeebli]

@hackdna let's check in on anything we will need to sync up to get forest working under 3.11. I checked last week and there was a requirements conflict (I believe with numpy) when I tried installing, but it may have been resolved by the celery update so I will have to check that again.

I'm making an issue on forest, I should be competent on this one when we get to it, but I think there are CI things that you usually handle.

[hydawo]

@biblicabeebli lets discuss this at our next meeting and how I or the RA's can help

[biblicabeebli]

This task got seriously delayed due to ios push notification resends issues dragging.
There is exactly one nice outcome - AWS finally added python 3.12 platform support....

[biblicabeebli]

@hackdna Do you know of any reason why the Forest codebase could not run under Python 3.12?

[hackdna]

Not at this time. If the test suite passes under 3.12 then it's likely there should be no issues.

[biblicabeebli]

Documentation has been added to the wiki at https://github.com/onnela-lab/beiwe-backend/wiki/Upgrading-Elastic-Beanstalk-from-Python-3.8-to-3.12

We are currently dogfooding the python3.12 branch on all data processing servers, our staging server has been fully migrated.

I will be merging the python3.12 branch into staged-updates this week. Other than some issues with our custom deploy hooks this transition went very smoothly.

[biblicabeebli]

This work is completed except for scouring for references to python 3.8.
Has not been merged to main for system administrator reasons.

@reyvababtista We skipped 3.11 and went to 3.12 because I'm just cruel that way. Let's check in on making sure the Docker container is functional and updated to staged-updates. We aren't intentionally using anything from 3.12 that wasn't in 3.11.