Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

more docs #1010

Merged
merged 3 commits into from
May 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/run_with_pixi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,11 @@ on:
- 'main'
paths-ignore:
- 'recipes/**'
- 'docs/**'
pull_request:
paths-ignore:
- 'recipes/**'
- 'docs/**'
jobs:

build_simple_with_pixi:
Expand Down
54 changes: 54 additions & 0 deletions docs/development/adding_packages.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,60 @@ requirements:
* [regex](https://github.com/emscripten-forge/recipes/tree/main/recipes/recipes_emscripten/regex)


### menson

For a meson package, the following requirements are usually needed.
```yaml
requirements:
build:
- ${{ compiler('cxx') }}
- cross-python_${{target_platform}}
- meson-python
- pip >=24
host:
- python
```
Furthermore a `emscripten.meson.cross` file is necessary to set the correct compiler and flags for the cross compilation.

```toml
# binaries section is at the end as may want to append python binary.

[properties]
needs_exe_wrapper = true
skip_sanity_check = true
longdouble_format = 'IEEE_QUAD_LE' # for numpy

[host_machine]
system = 'emscripten'
cpu_family = 'wasm32'
cpu = 'wasm'
endian = 'little'

[binaries]
exe_wrapper = 'node'
pkgconfig = 'pkg-config'

```

in the build script we append the python binary to the cross file and
pass the cross file to the pip command.

```bash
#!/bin/bash

cp $RECIPE_DIR/emscripten.meson.cross $SRC_DIR
echo "python = '${PYTHON}'" >> $SRC_DIR/emscripten.meson.cross

${PYTHON} -m pip install . -vvv --no-deps --no-build-isolation \
-Csetup-args="--cross-file=$SRC_DIR/emscripten.meson.cross"
```


**Example recipes**:

* [contourpy](https://github.com/emscripten-forge/recipes/tree/main/recipes/recipes_emscripten/contourpy)
* [numpy](https://github.com/emscripten-forge/recipes/tree/main/recipes/recipes_emscripten/numpy)

### rust
Building rust package with PyO3 / maturin works usually out of the box.

Expand Down
70 changes: 70 additions & 0 deletions docs/development/conda_build_config.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@

# Conda build config

Simmilar to conda-forge we globally pin packages.
This is essentialy a list of packages and versions like
```yaml
# ...
numpy:
- 1.25.2
occt:
- '7.5'
openblas:
- 0.3.*
openexr:
- 2.5
openjpeg:
- '2.4'
openmpi:
- 4
openssl:
- 1.1.1
openturns:
- '1.18'
orc:
- 1.7.2
# ...
```

This list is used to pin the versions of the dependencies in the `recipe.yaml` of recipes.

Therefore instead of writing

```yaml
requirements:
host:
- numpy 1.25.2
run:
- numpy 1.25.2
```

we can write

```yaml
requirements:
host:
- numpy
run:
- numpy

```
Furthermore this build-config contains the information which compiler to use for which platform.
While conda forge can be found [here](https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/conda_build_config.yaml),
we **need** to maintain our on [conda-build-config](https://github.com/emscripten-forge/recipes/blob/main/conda_build_config.yaml). In particular we need to setup the emscripten compiler.

!!! note
The conda-build-config of emscripten-forge uses the [rattler-recipies format](https://github.com/prefix-dev/rattler-build?tab=readme-ov-file#the-recipe-format)


```yaml
cxx_compiler:
- if: emscripten
then:
- emscripten
- if: linux
then:
- gxx
- if: osx
then:
- clangxx
```
22 changes: 22 additions & 0 deletions docs/development/troubleshooting.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Troubleshooting


Compiling for emscripten-wasm32 is a complex task. This section provides some tips and tricks to help you troubleshoot common issues.

## Buildtime errors

This section is still empty, feel free to contribute by opening a pull request.

## Runtime errors

### `PyLong_FromLongLong`: imported function does not match the expected type

When facing an error at runtime while importing a shared-library/ python-package like the following:
```
LinkError: WebAssembly.instantiate():
function="PyLong_FromLongLong":
imported function does not match the expected type
```
or similar error message containing the woring `*LongLong` or similar, it is likely
the linker flag `-s WASM_BIGINT` is missing.

14 changes: 10 additions & 4 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,19 +6,25 @@ Conda-forge does not (yet) support the `emscripten-wasm32` platform. `emscripten
The recipe repository not only stores the recipe, but also builds and upload the recipe
with github actions to the `emscripten-forge` channel on [quetz](https://beta.mamba.pm/channels/emscripten-forge)

##

# Table of Contents
**Development**:
### Development

* [Adding_packages](development/adding_packages)
* [Recipe format](development/recipe_format)
* [Local Builds](development/local_builds)
* [Conda build config](development/conda_build_config)
* [Troubleshooting](development/troubleshooting)

**Usage**:
### Usage

* [Installing packages](usage/installing_packages)
* [Jupyterlite](usage/jupyterlite)
* [Package server](usage/package_server)

**Project**:
### Project

* [Blog](blog)
* [Related Projects](project/related_projects)
* [FAQ](project/faq)
* [Credits](project/credits)
18 changes: 18 additions & 0 deletions docs/project/credits.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Credids

## Pyodide
This project is heavily inspired by the [pyodide](https://pyodide.org/en/stable/) project.
Many recipes, build scripts and patches are heavily inspired by the pyodide project.
Many thanks to the pyodide team for their work.

## Conda Forge

Many thanks to the [conda-forge](https://conda-forge.org/) team for their work.
Many recipes, build scripts and patches are heavily inspired by the conda-forge project.


## Contributors

This project has been started by [Thorsten Beier](https://github.com/derthorsten/) and [Wolf Wolfprecht](https://github.com/wolfv).
Sine then many contributors have joined the project.
Many thanks to [all the contributors](https://github.com/emscripten-forge/recipes/graphs/contributors) of the emscripten-forge project.
11 changes: 11 additions & 0 deletions docs/project/faq.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# FAQ

This section is relatively empty, feel free to contribute by opening a pull request.

## What is the difference between emscripten-forge and pyodide?

Pyodide is a full python distribution for `emscripten-wasm32` running in the browser.
Therefore all packages are either python packages or shared libraries needed by python packages.
Emscripten-forge on the other hand is a conda channel providing packages for the `emscripten-wasm32` platform.
This means there is a great overlap in the provided python packages, but emscripten-forge also provided non-python packages for the `emscripten-wasm32` platform.
Furthermore pyodide lives in the `pip` ecosystem, while emscripten-forge lives in the `conda/mamamba/rattler` ecosystem.
2 changes: 1 addition & 1 deletion docs/hosting.md → docs/usage/pacakge_server.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Quetz Server
# Packagve Server

Emscripten forge packages are hosted as
as a the [emscripten-forge](https://beta.mamba.pm/channels/emscripten-forge) channel on a [Quetz](https://quetz.io/) server.
Expand Down
Loading