Skip to content

Commit

Permalink
Minor edits in documentation (emscripten-forge#1044)
Browse files Browse the repository at this point in the history
* Fix typos

* Update link
  • Loading branch information
IsabelParedes authored and luizirber committed Jul 31, 2024
1 parent f29f01b commit 7be9686
Show file tree
Hide file tree
Showing 12 changed files with 72 additions and 71 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Emscripten forge
[![CI](https://img.shields.io/badge/emscripten_forge-docs-yellow)](https://emscripten-forge.org)
[![CI](https://img.shields.io/badge/emscripten_forge-blog-pink)](https://emscripten-forge.org)
[![CI](https://img.shields.io/badge/emscripten_forge-blog-pink)](https://emscripten-forge.org/blog/)

Visit [emscripten-forge.org](https://emscripten-forge.org) for more information and the documentation.
36 changes: 18 additions & 18 deletions docs/development/adding_packages.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,14 @@
# Adding packages

To add a new package to emscripten-forge, just create a Pull Request in this repository.
To add a new package to emscripten-forge, just create a Pull Request in [this repository](https://github.com/emscripten-forge/recipes/pulls).
The recipe format is described in the [rattler-build recipe format](https://github.com/prefix-dev/rattler-build?tab=readme-ov-file#the-recipe-format)


## C/C++ Packages
### CMake

Adding cmake based packages is easy. Usually it is enough to replace the `cmake` command with the `emcmake` command and
`make` with `emmake` (see the [emscripten documentation](https://emscripten.org/docs/compiling/Building-Projects.html#integrating-with-a-build-system) for more details)


`make` with `emmake` (see the [emscripten documentation](https://emscripten.org/docs/compiling/Building-Projects.html#integrating-with-a-build-system) for more details).

**Example recipes**:

Expand All @@ -20,19 +18,20 @@ Adding cmake based packages is easy. Usually it is enough to replace the `cmake`

### Configure / Make

Usually it is enough to replace the `./configure` command with the `emconfigure ./configure` (see the [emscripten documentation](https://emscripten.org/docs/compiling/Building-Projects.html#integrating-with-a-build-system) for more details)
Usually it is enough to replace the `./configure` command with the `emconfigure ./configure` (see the [emscripten documentation](https://emscripten.org/docs/compiling/Building-Projects.html#integrating-with-a-build-system) for more details).


## Python Packages

### pip / setuptools
For simple package only these requirements are usually necessary:
```yaml

For a simple package, only these requirements are usually necessary:

```yaml
requirements:
build:
- python
- cross-python_${{target_platform}}
- cross-python_${{ target_platform }}
- ${{ compiler("c") }}
- pip
host:
Expand All @@ -46,9 +45,10 @@ requirements:
* [regex](https://github.com/emscripten-forge/recipes/tree/main/recipes/recipes_emscripten/regex)
### menson
### meson
For a meson package, the following requirements are usually needed.
```yaml
requirements:
build:
Expand All @@ -59,10 +59,10 @@ requirements:
host:
- python
```
Furthermore a `emscripten.meson.cross` file is necessary to set the correct compiler and flags for the cross compilation.
Furthermore, an `emscripten.meson.cross` file is necessary to set the correct compiler and flags for cross compilation.

```toml
# binaries section is at the end as may want to append python binary.
# binaries section is at the end to be able to append the python binary.
[properties]
needs_exe_wrapper = true
Expand All @@ -81,8 +81,7 @@ pkgconfig = 'pkg-config'
```

in the build script we append the python binary to the cross file and
pass the cross file to the pip command.
In the build script, we append the python binary to the *.cross* file and pass this *.cross* file to the pip command.

```bash
#!/bin/bash
Expand All @@ -94,20 +93,21 @@ ${PYTHON} -m pip install . -vvv --no-deps --no-build-isolation \
-Csetup-args="--cross-file=$SRC_DIR/emscripten.meson.cross"
```


**Example recipes**:

* [contourpy](https://github.com/emscripten-forge/recipes/tree/main/recipes/recipes_emscripten/contourpy)
* [numpy](https://github.com/emscripten-forge/recipes/tree/main/recipes/recipes_emscripten/numpy)

### rust
Building rust package with PyO3 / maturin works usually out of the box.
### rust

Building rust packages with PyO3 / maturin typically works out of the box.

For a maturin / cffi / pyo3 package, the following requirements are usually necessary:

```yaml
requirements:
build:
- cross-python_${{target_platform}}
- cross-python_${{ target_platform }}
- cffi == 1.15.1 (# at the time of writing pinning cffi == 1.15.1 is necessary, may change in the future)
- setuptools-rust
- rust
Expand All @@ -116,7 +116,7 @@ requirements:
host:
- python
# at the time of writing pinning cffi == 1.15.1 is necessary, may change in the future)
- cffi == 1.15.1
- cffi == 1.15.1
run:
# at the time of writing pinning cffi == 1.15.1 is necessary, may change in the future)
- cffi == 1.15.1
Expand Down
19 changes: 10 additions & 9 deletions docs/development/conda_build_config.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@

# Conda build config

Simmilar to conda-forge we globally pin packages.
This is essentialy a list of packages and versions like
Similar to conda-forge, we globally pin packages.
This is essentially a list of packages and versions such as

```yaml
# ...
numpy:
Expand All @@ -28,7 +29,7 @@ orc:

This list is used to pin the versions of the dependencies in the `recipe.yaml` of recipes.

Therefore instead of writing
Therefore, instead of writing

```yaml
requirements:
Expand All @@ -48,14 +49,14 @@ requirements:
- numpy

```
Furthermore this build-config contains the information which compiler to use for which platform.
While conda forge can be found [here](https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/conda_build_config.yaml),
we **need** to maintain our on [conda-build-config](https://github.com/emscripten-forge/recipes/blob/main/conda_build_config.yaml). In particular we need to setup the emscripten compiler.
Furthermore, this build-config specifies which compiler to use for each platform.
While conda-forge build configuration can be found [here](https://github.com/conda-forge/conda-forge-pinning-feedstock/blob/main/recipe/conda_build_config.yaml),
we **need** to maintain our own [conda-build-config](https://github.com/emscripten-forge/recipes/blob/main/conda_build_config.yaml). In particular, we need to set up the emscripten compiler.

!!! note
The conda-build-config of emscripten-forge uses the [rattler-recipies format](https://github.com/prefix-dev/rattler-build?tab=readme-ov-file#the-recipe-format)
The conda-build-config of emscripten-forge uses the [rattler-recipes format](https://github.com/prefix-dev/rattler-build?tab=readme-ov-file#the-recipe-format)


```yaml
cxx_compiler:
- if: emscripten
Expand Down
13 changes: 7 additions & 6 deletions docs/development/local_builds.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,12 @@
## Local with `pixi`

To build a package locally, the easiest way is to use `pixi` (see [/pixi.sh/latest/#installation](https://pixi.sh/latest/#installation) for installation instructions).
Pixi will under the hood use `rattler-build` to build the package.
Under the hood, `pixi` uses `rattler-build` to build the package.

```bash
# this only needs to be done once
pixi run setup
pixi run setup

# this builds the package
pixi run build-emscripten-wasm32-pkg recipes/recipes_emscripten/regex
```
Expand All @@ -17,24 +18,24 @@ pixi run build-emscripten-wasm32-pkg recipes/recipes_emscripten/regex
We recommend using the `pixi` command to build packages locally. However, if you want to use `rattler-build` directly, you can do so with the following steps:


### create the environment
### Create the environment
Create a new conda environment from `ci_env.yml` and install playwright in this environment:
On a Linux / MacOS this can be done with:
```bash
micromamba create -n emscripten-forge -f ci_env.yml --yes
micromamba activate emscripten-forge
```
```


**All further steps should be executed in this environment.**
Ie if you open a new terminal, you have to activate the environment again with `micromamba activate emscripten-forge`.
I.e. if you open a new terminal, you have to activate the environment again with `micromamba activate emscripten-forge`.

### Setup emsdk

We currently need a patched version of emsdk. This is because emscripten had some regressions in the `3.1.45` release wrt. dynamic loading of shared libraries. We use the `./emsdk/setup_emsdk.sh` which takes
two arguments: the emsdk version and the path where emsdk should be installed.
In this example we choose `~/emsdk` as the installation path. You have to use version `3.1.45`.

```bash
./emsdk/setup_emsdk.sh 3.1.45 ~/emsdk
```
Expand Down
8 changes: 3 additions & 5 deletions docs/development/recipe_format.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
# Recipe format


Typically, a recipe is directory with a `recipe.yaml` file that contains the information needed to build a package
and optionally an additonal `build.sh` script that is executed during the build process.
Typically, a recipe is a directory with a `recipe.yaml` file that contains the information needed to build a package and optionally an additional `build.sh` script that is executed during the build process.

## recipe.yaml

Expand All @@ -18,7 +16,7 @@ package:

source:
# the url is formed from a "template" with the context variables
url: https://pypi.io/packages/source/r/${{name}}/${{name}}-${{ version }}.tar.gz
url: https://pypi.io/packages/source/r/${{ name }}/${{ name }}-${{ version }}.tar.gz
sha256: 97f32dc03a8054a4c4a5ab5d761ed4861e828b2c200febd4e46857069a483916

build:
Expand All @@ -27,7 +25,7 @@ build:
requirements:
build:
- python
- cross-python_${{target_platform}}
- cross-python_${{ target_platform }}
- ${{ compiler("c") }}
- pip
host:
Expand Down
7 changes: 3 additions & 4 deletions docs/development/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,11 @@ This section is still empty, feel free to contribute by opening a pull request.

### `PyLong_FromLongLong`: imported function does not match the expected type

When facing an error at runtime while importing a shared-library/ python-package like the following:
When facing an error at runtime while importing a shared-library / python-package like the following:
```
LinkError: WebAssembly.instantiate():
LinkError: WebAssembly.instantiate():
function="PyLong_FromLongLong":
imported function does not match the expected type
```
or similar error message containing the woring `*LongLong` or similar, it is likely
or an error message containing `*LongLong` or similar wording, it is likely that
the linker flag `-s WASM_BIGINT` is missing.

17 changes: 8 additions & 9 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,28 +3,27 @@

Emscripten-forge is a GitHub [organization](https://github.com/emscripten-forge)/[repository](https://github.com/emscripten-forge/recipes) containing [conda recipes](https://github.com/emscripten-forge/recipes) for the `emscripten-wasm32` platform.
Conda-forge does not (yet) support the `emscripten-wasm32` platform. `emscripten-forge` fills this gap by providing a channel with conda packages for the `emscripten-wasm32` platform.
The recipe repository not only stores the recipe, but also builds and upload the recipe
with github actions to the `emscripten-forge` channel on [quetz](https://beta.mamba.pm/channels/emscripten-forge)
The recipes repository not only stores the recipe files for multiple packages, but it also builds and uploads these packages to the `emscripten-forge` channel on [Quetz](https://beta.mamba.pm/channels/emscripten-forge)

##
##

### Development

* [Adding_packages](development/adding_packages)
* [Adding packages](development/adding_packages)
* [Recipe format](development/recipe_format)
* [Local Builds](development/local_builds)
* [Local builds](development/local_builds)
* [Conda build config](development/conda_build_config)
* [Troubleshooting](development/troubleshooting)

### Usage

* [Installing packages](usage/installing_packages)
* [Jupyterlite](usage/jupyterlite)
* [JupyterLite](usage/jupyterlite)
* [Package server](usage/package_server)

### Project
### Project

* [Blog](blog)
* [Related Projects](project/related_projects)
* [Related projects](project/related_projects)
* [FAQ](project/faq)
* [Credits](project/credits)
2 changes: 1 addition & 1 deletion docs/project/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ Pyodide is a full python distribution for `emscripten-wasm32` running in the bro
Therefore all packages are either python packages or shared libraries needed by python packages.
Emscripten-forge on the other hand is a conda channel providing packages for the `emscripten-wasm32` platform.
This means there is a great overlap in the provided python packages, but emscripten-forge also provided non-python packages for the `emscripten-wasm32` platform.
Furthermore pyodide lives in the `pip` ecosystem, while emscripten-forge lives in the `conda/mamamba/rattler` ecosystem.
Furthermore pyodide lives in the `pip` ecosystem, while emscripten-forge lives in the `conda/mamba/rattler` ecosystem.
10 changes: 5 additions & 5 deletions docs/project/related_projects.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,19 @@
## Pyodide

[Pyodide](https://pyodide.org/en/stable/index.html) is a Python distribution for the browser and Node.js based on WebAssembly.
Emscripten-forge would not have been possible without the pioneering work of the pyodide team.
Emscripten-forge would not have been possible without the pioneering work of the pyodide team.
Many aspects of this project are heavily inspired by the pyodide project. This includes the build scripts and many of the patches which have been taken from the pyodide packages.


## pyjs
[pyjs](https://emscripten-forge.github.io/pyjs/) is modern pybind11 + emscripten Embind based Python <-> JavaScript foreign function interface (FFI) for wasm/emscripten compiled Python.
The API is loosly based on the FFI of pyodide.
The API is loosely based on the FFI of pyodide.

## pyjs-code-runner
[pyjs-code-runner](https://github.com/emscripten-forge/pyjs-code-runner) is a "driver" to run python code in a wasm environment, almost like running vanilla python code. This is used to run the tests of the emscripten-forge packages.

## JupyterLite
[JupyterLite](https://jupyterlite.readthedocs.io/en/stable/) is a JupyterLab distribution that runs entirely in the browser built from the ground-up using JupyterLab components and extensions.
[JupyterLite](https://jupyterlite.readthedocs.io/en/stable/) is a JupyterLab distribution that runs entirely in the browser built from the ground-up using JupyterLab components and extensions.


## JupyterLite Xeus
Expand All @@ -30,8 +30,8 @@ and packages from emscripten-forge in a jupyterlite environment.

## pixi

[pixi](https://pixi.sh/latest/) is a package management tool for developers. It allows the developer to install libraries and applications in a reproducible way.
[pixi](https://pixi.sh/latest/) is a package management tool for developers. It allows the developer to install libraries and applications in a reproducible way.
Emscripten-forge uses pixi to setup the environment for building the packages.

## Quetz
Quetz is a [open source ](https://beta.mamba.pm/channels/emscripten-forge)conda package server. It is used to host the[ emscripten-forge](https://beta.mamba.pm/channels/emscripten-forge) packages.
Quetz is an [open source](https://beta.mamba.pm/channels/emscripten-forge) conda package server. It is used to host the [emscripten-forge](https://beta.mamba.pm/channels/emscripten-forge) packages.
4 changes: 2 additions & 2 deletions docs/usage/installing_packages.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#Installing packages
# Installing packages

## Install packages with micromamba
We recommend using micromamba to install packages from this channel.
Expand All @@ -15,4 +15,4 @@ micromamba create -n my-channel-name \
python numpy scipy matplotlib
```

All noarch packages are installed from the `conda-forge` channel, all packages compiled for the `emscripten-wasm32` platform are provided via the `emscripten-forge` channel.
All noarch packages are installed from the `conda-forge` channel, all packages compiled for the `emscripten-wasm32` platform are provided via the `emscripten-forge` channel.
20 changes: 12 additions & 8 deletions docs/usage/jupyterlite.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,16 +3,16 @@

## Installation

To consume emscripten-forge packages in JupyterLite environment, the `jupyterlite_xeus` package needs
to be installed. This can be done with `pip`
To consume emscripten-forge packages in a JupyterLite environment, the `jupyterlite_xeus` package needs
to be installed. This can be done with `pip`

```bash
pip install jupyterlite_xeus
```bash
pip install jupyterlite_xeus
```
or `conda`/`mamba`/`micromamba`
or `conda`/`mamba`/`micromamba`

```
mamba install jupyterlite_xeus
mamba install jupyterlite-xeus
```

## Usage
Expand All @@ -21,7 +21,7 @@ mamba install jupyterlite_xeus

Emscripten-forge provides xeus kernels for multiple languages, this document focuses on the Python kernel, namely `xeus-python`.
While the other kernels can also be installed as described below, adding custom packages is only supported for the `xeus-python` kernel
at the moment.
at the moment.

### From environment file

Expand All @@ -46,7 +46,7 @@ jupyter lite build --XeusAddon.environment_file=some_path/to/environment.yaml
Create a environment with the desired packages. Here is an example with numpy as a additional dependency

```bash
micromamba create
micromamba create
-n myenv \
--platform=emscripten-wasm32 \
-c https://repo.mamba.pm/emscripten-forge \
Expand All @@ -55,4 +55,8 @@ micromamba create
"python>=3.11" numpy pandas xeus-python
```

Use the following command to build JupyterLite.

```sh
jupyter lite build --XeusAddon.prefix=$MAMBA_ROOT_PREFIX/envs/myenv
```
Loading

0 comments on commit 7be9686

Please sign in to comment.