Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Emscripten build and test on GitHub Actions #1026

Closed
wants to merge 10 commits into from
Closed
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 68 additions & 0 deletions .github/workflows/emscripten.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
name: Emscripten build and test

on:
workflow_dispatch:
pull_request:
push:
branches: [main]

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

jobs:
build:
runs-on: ubuntu-latest
timeout-minutes: 30

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Setup Python
uses: actions/setup-python@v4
with:
python-version: 3.11

- name: Install pyodide-build
run: |
pip install pyodide-build
echo EMSCRIPTEN_VERSION=$(pyodide config get emscripten_version) >> $GITHUB_ENV

- name: Setup Emscripten
uses: mymindstorm/setup-emsdk@v12
with:
version: ${{ env.EMSCRIPTEN_VERSION }}

- name: Build
run: pyodide build

- name: Setup node
uses: actions/setup-node@v4
with:
node-version: 20

# clone awkward and build a wheel, copy the wheel
- name: Awkward wheel
run: |
git clone https://github.com/scikit-hep/awkward.git
cd awkward
git checkout main
git submodule update --init --recursive
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
git submodule update --init --recursive
git submodule update --init --recursive
pipx run nox -s prepare -- --headers --signatures --tests

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

n.b. you'll still need awkward's C++ components too

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My initial goal was to test the build (only uproot) and run the tests. However the uproot build will only work if the dependencies (awkward) are available, and there is a significant delay between the publication in pypi and it's availability from pyodide, this means that this CI will fail for a while whenever there is a release that modifies dependencies.

To test the CI job I though of building the awkward (and awkward-cpp) wheel in this CI job (in particular the version of awkward listed in pyproject.toml).

I am having some troubles understanding the necessary steps though.

pyodide build
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

pyodide build only builds the pure-Python component, not the C++ build.

mv dist/*.whl ../dist

- name: Set up Pyodide virtual environment
run: |
pyodide venv .venv-pyodide
source .venv-pyodide/bin/activate
# python -m pip install .[test]
python -m pip install pytest
ls -l dist
python -m pip install dist/*.whl
python -c "import sys; print(sys.platform)"

- name: Test
run: |
source .venv-pyodide/bin/activate
python -m pytest -vv tests -m emscripten --reruns 3 --reruns-delay 30 --only-rerun "(?i)http|timeout|connection"
5 changes: 3 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ classifiers = [
"Topic :: Utilities"
]
dependencies = [
"awkward>=2.4.6",
"awkward>=2.5.0",
"importlib-metadata;python_version<\"3.8\"",
"numpy",
"packaging",
Expand Down Expand Up @@ -94,7 +94,8 @@ log_cli_level = "info"
markers = [
"slow",
"network",
"xrootd"
"xrootd",
"emscripten"
]
minversion = "6.0"
testpaths = ["tests"]
Expand Down
4 changes: 1 addition & 3 deletions tests/test_0014-all-ttree-versions.py
Original file line number Diff line number Diff line change
@@ -1,16 +1,14 @@
# BSD 3-Clause License; see https://github.com/scikit-hep/uproot5/blob/main/LICENSE

import json
import sys
from io import StringIO

import numpy
import pytest
import skhep_testdata

import uproot


@pytest.mark.emscripten
def test_streamerless_read():
with uproot.open(
skhep_testdata.data_path("uproot-sample-5.23.02-uncompressed.root")
Expand Down