Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support Docker installation #140

Merged
merged 3 commits into from
Feb 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
# Copyright (c) 2023 Amphion.
#
# This source code is licensed under the MIT license found in the
# LICENSE file in the root directory of this source tree.

# Other version: https://hub.docker.com/r/nvidia/cuda/tags
FROM nvidia/cuda:11.8.0-cudnn8-devel-ubuntu18.04

ARG DEBIAN_FRONTEND=noninteractive
ARG PYTORCH='2.0.0'
ARG CUDA='cu118'
ARG SHELL='/bin/bash'
ARG MINICONDA='Miniconda3-py39_23.3.1-0-Linux-x86_64.sh'

ENV LANG=en_US.UTF-8 PYTHONIOENCODING=utf-8 PYTHONDONTWRITEBYTECODE=1 CUDA_HOME=/usr/local/cuda CONDA_HOME=/opt/conda SHELL=${SHELL}
ENV PATH=$CONDA_HOME/bin:$CUDA_HOME/bin:$PATH \
LD_LIBRARY_PATH=$CUDA_HOME/lib64:$LD_LIBRARY_PATH \
LIBRARY_PATH=$CUDA_HOME/lib64:$LIBRARY_PATH \
CONDA_PREFIX=$CONDA_HOME \
NCCL_HOME=$CUDA_HOME

# Install ubuntu packages
RUN sed -i 's/archive.ubuntu.com/mirrors.cloud.tencent.com/g' /etc/apt/sources.list \
&& sed -i 's/security.ubuntu.com/mirrors.cloud.tencent.com/g' /etc/apt/sources.list \
&& rm /etc/apt/sources.list.d/cuda.list \
&& apt-get update \
&& apt-get -y install \
python3-pip ffmpeg git less wget libsm6 libxext6 libxrender-dev \
build-essential cmake pkg-config libx11-dev libatlas-base-dev \
libgtk-3-dev libboost-python-dev vim libgl1-mesa-glx \
libaio-dev software-properties-common tmux \
espeak-ng

# Install miniconda with python 3.9
USER root
# COPY Miniconda3-py39_23.3.1-0-Linux-x86_64.sh /root/anaconda.sh
RUN wget -t 0 -c -O /tmp/anaconda.sh https://repo.anaconda.com/miniconda/${MINICONDA} \
&& mv /tmp/anaconda.sh /root/anaconda.sh \
&& ${SHELL} /root/anaconda.sh -b -p $CONDA_HOME \
&& rm /root/anaconda.sh

RUN conda create -y --name amphion python=3.9.15

WORKDIR /app
COPY env.sh env.sh
RUN chmod +x ./env.sh

RUN ["conda", "run", "-n", "amphion", "-vvv", "--no-capture-output", "./env.sh"]

RUN conda init \
&& echo "\nconda activate amphion\n" >> ~/.bashrc

CMD ["/bin/bash"]

# *** Build ***
# docker build -t realamphion/amphion .

# *** Run ***
# cd Amphion
# docker run --runtime=nvidia --gpus all -it -v .:/app -v /mnt:/mnt_host realamphion/amphion

# *** Push and release ***
# docker login
# docker push realamphion/amphion
19 changes: 19 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,10 @@ Amphion unifies the data preprocess of the open-source datasets including [Audio

## πŸ“€ Installation

Amphion can be installed through either Setup Installer or Docker Image.

### Setup Installer

```bash
git clone https://github.com/open-mmlab/Amphion.git
cd Amphion
Expand All @@ -93,6 +97,21 @@ conda activate amphion
sh env.sh
```

### Docker Image

1. Install [Docker](https://docs.docker.com/get-docker/), [NVIDIA Driver](https://www.nvidia.com/download/index.aspx), [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html), and [CUDA](https://developer.nvidia.com/cuda-downloads).

2. Run the following commands:
```bash
git clone https://github.com/open-mmlab/Amphion.git
cd Amphion

docker pull realamphion/amphion
docker run --runtime=nvidia --gpus all -it -v .:/app realamphion/amphion
```
Mount dataset by argument `-v` is necessary when using Docker. Please refer to [Mount dataset in Docker container](egs/datasets/docker.md) and [Docker Docs](https://docs.docker.com/engine/reference/commandline/container_run/#volume) for more details.


## 🐍 Usage in Python

We detail the instructions of different tasks in the following recipes:
Expand Down
2 changes: 2 additions & 0 deletions egs/datasets/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ Amphion support the following academic datasets (sort alphabetically):

The downloading link and the file structure tree of each dataset is displayed as follows.

> **Note:** When using Docker to run Amphion, mount the dataset to the container is necessary after downloading. Check [Mount dataset in Docker container](./docker.md) for more details.

## AudioCaps

AudioCaps is a dataset of around 44K audio-caption pairs, where each audio clip corresponds to a caption with rich semantic information.
Expand Down
19 changes: 19 additions & 0 deletions egs/datasets/docker.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Mount dataset in Docker container

When using Docker to run Amphion, mount the dataset to the container first is needed. It is recommend to mounte dataset to `/mnt/<dataset_name>` in the container, where `<dataset_name>` is the name of the dataset.

When configuring the dataset in `exp_config.json`, you should use the path `/mnt/<dataset_name>` as the dataset path instead of the actual path on your host machine. Otherwise, the dataset will not be found in the container.

## Mount Example

```bash
docker run --runtime=nvidia --gpus all -it -v .:/app -v <dataset_path1>:/mnt/<dataset_name1> -v <dataset_path2>:/mnt/<dataset_name2> amphion
```

For example, if you want to use the `LJSpeech` dataset, you can mount the dataset to `/mnt/LJSpeech` in the container.

```bash
docker run --runtime=nvidia --gpus all -it -v .:/app -v /home/username/datasets/LJSpeech:/mnt/LJSpeech amphion
```

If you want to use multiple datasets, you can mount them to different directories in the container by adding more `-v` options.
3 changes: 3 additions & 0 deletions env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@
# This source code is licensed under the MIT license found in the
# LICENSE file in the root directory of this source tree.

# Raise error if any command fails
set -e

# Install ffmpeg in Linux
conda install -c conda-forge ffmpeg

Expand Down
Loading