From a7ccafa52fe888d6c4771aded43216e18b5e64a4 Mon Sep 17 00:00:00 2001 From: Alex Valcourt Caron Date: Thu, 17 Aug 2023 15:50:52 -0400 Subject: [PATCH 1/2] First draft at devcontainers documentation + mini readme update --- README.md | 12 ++++- source/index.rst | 1 + source/intro_to/explore_software.rst | 2 + source/intro_to/explore_virtual_machines.rst | 2 + source/our_tools/development_containers.rst | 46 ++++++++++++++++++++ 5 files changed, 61 insertions(+), 2 deletions(-) create mode 100644 source/our_tools/development_containers.rst diff --git a/README.md b/README.md index c8fd1f2..4ae762c 100644 --- a/README.md +++ b/README.md @@ -4,9 +4,17 @@ Here is the link to our web site: https://scil-documentation.readthedocs.io/ +# Developpers -*Note for developers: To test your changes, you can build the website locally with: +The documententation is generated using [Sphinx](https://www.sphinx-doc.org/en/master/). +Here is how to install the required Python dependencies in your interpreter : + +``` +pip install sphinx sphinx-rtd-theme +``` + +Then, to test your changes, build the website locally with: make html -Then go to scil-documentation/build/html/ and click on index.html. +The website can be accessed by clicking on **index.html** in the **scil-documentation/build/html/** directory. diff --git a/source/index.rst b/source/index.rst index 888f2d9..275a970 100644 --- a/source/index.rst +++ b/source/index.rst @@ -38,6 +38,7 @@ Here, you will find a summary of the tools we develop, and some clues on how we :caption: Our tools our_tools/scilpy + our_tools/development_containers our_tools/tractoflow our_tools/recobundles our_tools/other_pipelines diff --git a/source/intro_to/explore_software.rst b/source/intro_to/explore_software.rst index 561bc6d..dcde0ea 100644 --- a/source/intro_to/explore_software.rst +++ b/source/intro_to/explore_software.rst @@ -32,6 +32,8 @@ Related links - Video tutorial `[link] `__ - Conference abstract `[link] `__ +.. _ref_scil_dependencies: + FSL --- diff --git a/source/intro_to/explore_virtual_machines.rst b/source/intro_to/explore_virtual_machines.rst index bc29cd9..b3f0dc1 100644 --- a/source/intro_to/explore_virtual_machines.rst +++ b/source/intro_to/explore_virtual_machines.rst @@ -78,6 +78,8 @@ This means there is a complexity to the installation, and that Docker is general :bash:`docker stop`: This command is used to stop a running container. It takes the container ID or name as an argument, and can be used to gracefully shut down the container. +.. _ref_docker_installation: + Installations ^^^^^^^^^^^^^ diff --git a/source/our_tools/development_containers.rst b/source/our_tools/development_containers.rst new file mode 100644 index 0000000..e8c16b1 --- /dev/null +++ b/source/our_tools/development_containers.rst @@ -0,0 +1,46 @@ +Development containers +====================== + +To streamline the execution of the various technologies the lab develops on a wide range of computing architectures and operating systems, and to facilitate their distribution to collaborators, we use Docker images. An image is a stand-alone, executable package that includes everything needed to run a collection of softwares, including the code, runtimes, libraries, environment variables, and configuration files. In Docker, images are used to create containers, isolated and portable environments that can be run on any computer with Docker installed, regardless of the operating system running on the computer. + +Two specialized images are developed for the lab and its collaborators: +* [Scilus](https://hub.docker.com/r/scilus/scilus/): contains the :ref:`main dependencies ` for the lab's projects, [Scilpy](https://github.com/scilus/scilpy) and [dmriqcpy](https://github.com/scilus/dmriqcpy) +* [Scilus-flows](https://hub.docker.com/r/scilus/scilus-flows/): contains :ref:`Nextflow ` and most pipelines developed by the lab, in addition to the content of Scilus. + +.. _ref_dev_with_docker: + +Developing with Docker +---------------------- + +To install Docker on your system, refer to the procedure :ref:`(here) `. Once Docker is installed, you can pull the Scilus image at a specific ** from the Docker Hub by running the following command: + +.. code-block:: bash + + docker pull scilus/scilus: + +To open a bash shell in container from the Scilus image, you can use the following command: + +.. code-block:: bash + + docker run --rm -it scilus/scilus: /bin/bash + +The ``--rm`` option will remove the container once you exit the shell. The ``-it`` options will open an interactive shell in the container, allowing you to launch commands. The ``/bin/bash`` command will open a bash shell in the container. You can then run any command available in the container, such as ``python``, ``git``, ``nextflow``, etc. + +.. note:: + + The ``--rm`` option is useful to avoid dangling containers on your system taking resources. However, if you want to keep the container, you can remove the ``--rm`` option. You can get the container ID using the ``docker ps`` command and attach to it using ``docker attach ``. You can remove the container later with ``docker rm ``. + +Docker in Visual Studio code +---------------------------- + +[Visual Studio Code](https://code.visualstudio.com/) is one of the IDE used by the lab. It is a free and open-source code editor developed by Microsoft. It includes support for debugging, embedded Git control, syntax highlighting, intelligent code completion, snippets, and code refactoring. It is also customizable, allowing users to change the theme, keyboard shortcuts, preferences, and install extensions that add additional functionality. + +Visual Studio Code has a [Docker extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) that allows you to develop inside a container. This is useful to avoid installing all the dependencies on your system and to ensure that the code will run in the same environment as the one used by the lab. To use this extension with the lab images, you need first to install Docker on your system and pull the Scilus image as described :ref:`above `. + +Once done, open a folder in Visual Studio Code and click on the Docker icon in the left menu. This will open the Docker tab, listing containers, images available locally and in connected remote registries, and much more. + +Click on the ``+`` icon in the *containers* pane and select ``Open current folder in container``. Choose a configuration from the list of base (Ubuntu is a good default one), the version of the base OS and additional features if desired, to generate a template configuration file. This will create a ``.devcontainer`` folder in your project with a ``devcontainer.json`` file. You can then edit this file to specify the image to use, the user to use in the container, the extensions to install, etc. To get an idea of a configuration for the lab, you can refer to the ``.devcontainer`` folder in the [Scilpy repository](https://github.com/scilus/scilpy). + +Once the configuration is done, click on the ``blue icon`` on the bottom left of the window and select ``Reopen in Container``. This will build the container and open a new window in the container. + +You can then open a terminal in the container and run any command available in the container, such as ``python``, ``git``, ``nextflow``, etc. All files and directories in the project folder will be available in the container, allowing you to edit them in Visual Studio Code. You will also have access to all the dependencies installed in the container, pre-configured with the lab's settings. From 7fddb9cbb9f6c6cdcd9acaaa32df3517db341921 Mon Sep 17 00:00:00 2001 From: Alex Valcourt Caron Date: Thu, 17 Aug 2023 18:34:09 -0400 Subject: [PATCH 2/2] uniformize rst --- source/our_tools/development_containers.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/source/our_tools/development_containers.rst b/source/our_tools/development_containers.rst index e8c16b1..81f26d3 100644 --- a/source/our_tools/development_containers.rst +++ b/source/our_tools/development_containers.rst @@ -4,8 +4,8 @@ Development containers To streamline the execution of the various technologies the lab develops on a wide range of computing architectures and operating systems, and to facilitate their distribution to collaborators, we use Docker images. An image is a stand-alone, executable package that includes everything needed to run a collection of softwares, including the code, runtimes, libraries, environment variables, and configuration files. In Docker, images are used to create containers, isolated and portable environments that can be run on any computer with Docker installed, regardless of the operating system running on the computer. Two specialized images are developed for the lab and its collaborators: -* [Scilus](https://hub.docker.com/r/scilus/scilus/): contains the :ref:`main dependencies ` for the lab's projects, [Scilpy](https://github.com/scilus/scilpy) and [dmriqcpy](https://github.com/scilus/dmriqcpy) -* [Scilus-flows](https://hub.docker.com/r/scilus/scilus-flows/): contains :ref:`Nextflow ` and most pipelines developed by the lab, in addition to the content of Scilus. +* `Scilus `__: contains the :ref:`main dependencies `__ for the lab's projects, `Scilpy `__ and `dmriqcpy `__ +* `Scilus-flows `__: contains :ref:`Nextflow ` and most pipelines developed by the lab, in addition to the content of Scilus. .. _ref_dev_with_docker: @@ -33,13 +33,13 @@ The ``--rm`` option will remove the container once you exit the shell. The ``-it Docker in Visual Studio code ---------------------------- -[Visual Studio Code](https://code.visualstudio.com/) is one of the IDE used by the lab. It is a free and open-source code editor developed by Microsoft. It includes support for debugging, embedded Git control, syntax highlighting, intelligent code completion, snippets, and code refactoring. It is also customizable, allowing users to change the theme, keyboard shortcuts, preferences, and install extensions that add additional functionality. +`Visual Studio Code `__ is one of the IDE used by the lab. It is a free and open-source code editor developed by Microsoft. It includes support for debugging, embedded Git control, syntax highlighting, intelligent code completion, snippets, and code refactoring. It is also customizable, allowing users to change the theme, keyboard shortcuts, preferences, and install extensions that add additional functionality. -Visual Studio Code has a [Docker extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) that allows you to develop inside a container. This is useful to avoid installing all the dependencies on your system and to ensure that the code will run in the same environment as the one used by the lab. To use this extension with the lab images, you need first to install Docker on your system and pull the Scilus image as described :ref:`above `. +Visual Studio Code has a `Dev containers extension `__ that allows you to develop inside a container. This is useful to avoid installing all the dependencies on your system and to ensure that the code will run in the same environment as the one used by the lab. To use this extension with the lab images, you need first to install Docker on your system and pull the Scilus image as described :ref:`above `. Once done, open a folder in Visual Studio Code and click on the Docker icon in the left menu. This will open the Docker tab, listing containers, images available locally and in connected remote registries, and much more. -Click on the ``+`` icon in the *containers* pane and select ``Open current folder in container``. Choose a configuration from the list of base (Ubuntu is a good default one), the version of the base OS and additional features if desired, to generate a template configuration file. This will create a ``.devcontainer`` folder in your project with a ``devcontainer.json`` file. You can then edit this file to specify the image to use, the user to use in the container, the extensions to install, etc. To get an idea of a configuration for the lab, you can refer to the ``.devcontainer`` folder in the [Scilpy repository](https://github.com/scilus/scilpy). +Click on the ``+`` icon in the *containers* pane and select ``Open current folder in container``. Choose a configuration from the list of base (Ubuntu is a good default one), the version of the base OS and additional features if desired, to generate a template configuration file. This will create a ``.devcontainer`` folder in your project with a ``devcontainer.json`` file. You can then edit this file to specify the image to use, the user to use in the container, the extensions to install, etc. To get an idea of a configuration for the lab, you can refer to the ``.devcontainer`` folder in the `Scilpy repository `__. Once the configuration is done, click on the ``blue icon`` on the bottom left of the window and select ``Reopen in Container``. This will build the container and open a new window in the container.