Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade python packages using poetry #1881

Merged
merged 13 commits into from
Feb 13, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .env
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ OPENC3_ENTERPRISE_REGISTRY=ghcr.io
OPENC3_ENTERPRISE_NAMESPACE=openc3
OPENC3_UBI_REGISTRY=registry1.dso.mil
OPENC3_UBI_IMAGE=ironbank/redhat/ubi/ubi8-minimal
OPENC3_UBI_TAG=8.8
OPENC3_UBI_TAG=8.10
# Defined here as blank to avoid warnings. Used in the compose.yaml to pass '-ubi'.
OPENC3_IMAGE_SUFFIX=
# Bucket & Volume configuration
Expand Down
14 changes: 7 additions & 7 deletions .github/workflows/api_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -107,18 +107,18 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install -r requirements-dev.txt
pip install poetry
poetry install
working-directory: openc3/python
- name: Lint with ruff
run: |
ruff --config=../openc3/python/pyproject.toml --format=github scripts/*.py
working-directory: openc3-cosmos-script-runner-api
poetry run ruff check --output-format=github ../../openc3-cosmos-script-runner-api/scripts
working-directory: openc3/python
- name: Run unit tests
run: |
coverage run -m pytest ./test/
coverage xml -i
working-directory: openc3-cosmos-script-runner-api
poetry run coverage run -m pytest ../../openc3-cosmos-script-runner-api/test/
poetry run coverage xml -i
working-directory: openc3/python
- uses: codecov/codecov-action@v5
with:
working-directory: openc3/python
Expand Down
59 changes: 30 additions & 29 deletions .github/workflows/cli.yml
Original file line number Diff line number Diff line change
Expand Up @@ -116,35 +116,36 @@ jobs:
./openc3.sh cli script run --wait 20 INST/procedures/checks.rb | tee /dev/tty | grep -q "script failed"
# run a script that will complete successfully
./openc3.sh cli script run INST/procedures/stash.rb | tee /dev/tty | grep "script complete"
- name: openc3.sh util save,load
shell: 'script -q -e -c "bash {0}"'
run: |
set -e
./openc3.sh util save docker.io openc3inc latest
# Verify the images were saved
ls tmp | grep openc3-cosmos-cmd-tlm-api-latest.tar
ls tmp | grep openc3-cosmos-script-runner-api-latest.tar
ls tmp | grep openc3-cosmos-init-latest.tar
ls tmp | grep openc3-operator-latest.tar
ls tmp | grep openc3-minio-latest.tar
ls tmp | grep openc3-redis-latest.tar
ls tmp | grep openc3-traefik-latest.tar
docker image rm openc3inc/openc3-cosmos-cmd-tlm-api:latest
docker image rm openc3inc/openc3-cosmos-script-runner-api:latest
docker image rm openc3inc/openc3-cosmos-init:latest
docker image rm openc3inc/openc3-operator:latest
docker image rm openc3inc/openc3-minio:latest
docker image rm openc3inc/openc3-redis:latest
docker image rm openc3inc/openc3-traefik:latest
./openc3.sh util load
# Verify the images were loaded
docker image ls | grep openc3-cosmos-cmd-tlm-api
docker image ls | grep openc3-cosmos-script-runner-api
docker image ls | grep openc3-cosmos-init
docker image ls | grep openc3-operator
docker image ls | grep openc3-minio
docker image ls | grep openc3-redis
docker image ls | grep openc3-traefik
# Removing due to write tmp/.docker_temp_3200699358: no space left on device
# - name: openc3.sh util save,load
# shell: 'script -q -e -c "bash {0}"'
# run: |
# set -e
# ./openc3.sh util save docker.io openc3inc latest
# # Verify the images were saved
# ls tmp | grep openc3-cosmos-cmd-tlm-api-latest.tar
# ls tmp | grep openc3-cosmos-script-runner-api-latest.tar
# ls tmp | grep openc3-cosmos-init-latest.tar
# ls tmp | grep openc3-operator-latest.tar
# ls tmp | grep openc3-minio-latest.tar
# ls tmp | grep openc3-redis-latest.tar
# ls tmp | grep openc3-traefik-latest.tar
# docker image rm openc3inc/openc3-cosmos-cmd-tlm-api:latest
# docker image rm openc3inc/openc3-cosmos-script-runner-api:latest
# docker image rm openc3inc/openc3-cosmos-init:latest
# docker image rm openc3inc/openc3-operator:latest
# docker image rm openc3inc/openc3-minio:latest
# docker image rm openc3inc/openc3-redis:latest
# docker image rm openc3inc/openc3-traefik:latest
# ./openc3.sh util load
# # Verify the images were loaded
# docker image ls | grep openc3-cosmos-cmd-tlm-api
# docker image ls | grep openc3-cosmos-script-runner-api
# docker image ls | grep openc3-cosmos-init
# docker image ls | grep openc3-operator
# docker image ls | grep openc3-minio
# docker image ls | grep openc3-redis
# docker image ls | grep openc3-traefik
- name: Cleanup volumes
shell: 'script -q -e -c "bash {0}"'
run: |
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/python_unit_tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,17 +35,17 @@ jobs:
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
pip install -r requirements-dev.txt
pip install poetry
poetry install
working-directory: openc3/python
- name: Lint with ruff
run: |
ruff --format=github openc3
poetry run ruff check --output-format=github openc3
working-directory: openc3/python
- name: Run unit tests
run: |
coverage run -m pytest ./test/
coverage xml -i
poetry run coverage run -m pytest ./test/
poetry run coverage xml -i
working-directory: openc3/python
- uses: codecov/codecov-action@v5
with:
Expand Down
8 changes: 4 additions & 4 deletions docs.openc3.com/docs/development/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,10 +75,10 @@ Code coverage reports can be found at `cosmos/openc3/coverage/index.html`
1. Navigate to **cosmos/openc3/python** folder. Run the command:

```bash
cosmos/openc3/python % python -m pip install -r requirements-dev.txt
cosmos/openc3/python % python -m pip install -r requirements.txt
cosmos/openc3/python % coverage run -m pytest
cosmos/openc3/python % coverage html
cosmos/openc3/python % python -m pip install poetry
cosmos/openc3/python % poetry install
cosmos/openc3/python % poetry run coverage run -m pytest
cosmos/openc3/python % poetry run coverage html
```

Code coverage reports can be found at `cosmos/openc3/python/coverage/index.html`
4 changes: 4 additions & 0 deletions mise.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[tools]
node = "20.10"
python = "3.10"
ruby = "3.2"
Original file line number Diff line number Diff line change
@@ -1 +1 @@
numpy==2.1.1
numpy==2.2.2

This file was deleted.

21 changes: 10 additions & 11 deletions openc3-cosmos-script-runner-api/scripts/run_script.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Copyright 2024 OpenC3, Inc.
# Copyright 2025 OpenC3, Inc.
# All Rights Reserved.
#
# This program is free software; you can modify and/or redistribute it
Expand All @@ -18,28 +18,27 @@
import time
import json
import sys
import traceback
from datetime import datetime, timezone
from openc3.script import get_overrides
from openc3.utilities.bucket import Bucket
from openc3.utilities.store import Store, EphemeralStore
from openc3.utilities.extract import convert_to_value
from openc3.utilities.logger import Logger
from openc3.environment import *
import traceback
from openc3.environment import OPENC3_CONFIG_BUCKET
from running_script import RunningScript, running_script_anycable_publish

start_time = time.time()

from running_script import RunningScript, running_script_anycable_publish

# # Load the bucket client code to ensure we authenticate outside ENV vars
# Load the bucket client code to ensure we authenticate outside ENV vars
Bucket.getClient()

del os.environ["OPENC3_BUCKET_USERNAME"]
del os.environ["OPENC3_BUCKET_PASSWORD"]
os.unsetenv("OPENC3_BUCKET_USERNAME")
os.unsetenv("OPENC3_BUCKET_PASSWORD")

# # Preload Store and remove Redis secrets from ENV
# Preload Store and remove Redis secrets from ENV
Store.instance()
EphemeralStore.instance()

Expand Down Expand Up @@ -73,7 +72,7 @@ def run_script_log(id, message, color="BLACK", message_log=True):
RunningScript.message_log().write(line_to_write + "\n", True)
running_script_anycable_publish(
f"running-script-channel:{id}",
{"type": "output", "line": line_to_write, "color": color}
{"type": "output", "line": line_to_write, "color": color},
)


Expand Down Expand Up @@ -128,7 +127,7 @@ def run_script_log(id, message, color="BLACK", message_log=True):
"type": "start",
"filename": path,
"active_scripts": len(running),
}
},
)

# Subscribe to the ActionCable generated topic which is namedspaced with channel_prefix
Expand Down Expand Up @@ -237,7 +236,7 @@ def run_script_log(id, message, color="BLACK", message_log=True):
"type": "script",
"method": "backtrace",
"args": running_script.current_backtrace,
}
},
)
case "debug":
run_script_log(
Expand Down Expand Up @@ -281,7 +280,7 @@ def run_script_log(id, message, color="BLACK", message_log=True):
)
running_script_anycable_publish(
"all-scripts-channel",
{"type": "complete", "active_scripts": active_scripts}
{"type": "complete", "active_scripts": active_scripts},
)
finally:
if running_script:
Expand Down
Loading
Loading