Skip to content

Commit

Permalink
Add example launch script to documentation.
Browse files Browse the repository at this point in the history
[skip ci]
  • Loading branch information
ethanluoyc committed Oct 19, 2023
1 parent 949e620 commit b6ca5af
Show file tree
Hide file tree
Showing 2 changed files with 72 additions and 3 deletions.
39 changes: 37 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,9 +30,44 @@ pip install git+https://github.com/ethanluoyc/lxm3
```

## Documentation
Since lxm3 implements the XManager API, you should get familiar with the concepts in the [XManager](https://github.com/deepmind/xmanager). Once you are familiar with the concepts, checkout the [examples/](examples/) directory for a quick start guide.
At a high level you can launch experiment by creating a launch script
called `launcher.py` that looks like:

```python
with xm_cluster.create_experiment(experiment_title="hello world") as experiment:
# Launch on a Slurm cluster
executor = xm_cluster.Slurm()
# or, if you want to use SGE:
# executor = xm_cluster.GridEngine()
# or, if you want to run locally:
# executor = xm_cluster.Local()

spec = xm_cluster.PythonPackage(
path=".",
entrypoint=xm_cluster.ModuleName("my_package.main"),
)

# package your code
[executable] = experiment.package(
[xm.Packageable(spec, executor_spec=executor.Spec())]
)

# add jobs to your experiment
experiment.add(
xm.Job(executable=executable, executor=executor)
)
```
and launch the experimet from the command line with
```python
lxm3 launch launcher.py
```

Many things happen under the hood. Since lxm3 implements the XManager
API, you should get familiar with the concepts in the
[XManager](https://github.com/deepmind/xmanager). Once you are
familiar with the concepts, checkout the [examples/](examples/)
directory for a quick start guide.

TODO: Add a simple example for usage.

## Components
lxm3 provides the following executable specification and executors.
Expand Down
36 changes: 35 additions & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,41 @@ Welcome to LXM3's documentation!
LXM3 provides an implementation for DeepMind's
`XManager <https://github.com/deepmind/xmanager/tree/main>`_ launch API
that aims to provide a similar experience for running experiments on
traditional HPC.
traditional HPC (SGE, Slurm).

With LXM3, launching your job on a HPC cluster is as simple as
creating a launch script, e.g. ``launcher.py``:

.. code-block:: python
with xm_cluster.create_experiment(experiment_title="Hello World") as experiment:
executor = xm_cluster.Slurm()
# or, if you want to use SGE:
# executor = xm_cluster.GridEngine()
# or, if you want to run locally:
# executor = xm_cluster.Local()
spec = xm_cluster.PythonPackage(
path=".",
entrypoint=xm_cluster.ModuleName("my_package.main"),
)
# package your code
[executable] = experiment.package(
[xm.Packageable(spec, executor_spec=executor.Spec())]
)
# add jobs to your experiment
experiment.add(
xm.Job(executable=executable, executor=executor)
)
and run the command

.. code-block::
lxm3 launch launcher.py
.. toctree::
:maxdepth: 2
Expand Down

0 comments on commit b6ca5af

Please sign in to comment.