Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explain how to get vg on the PATH #4297

Merged
merged 5 commits into from
May 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 67 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,27 +55,41 @@ The easiest way to get vg is to download one of our release builds for Linux. We

If you don't want to or can't use a pre-built release of vg, or if you want to become a vg developer, you can build it from source instead.

#### Linux: Clone VG

First, obtain the repo and its submodules:

git clone --recursive https://github.com/vgteam/vg.git
cd vg

#### Linux: Install Dependencies

Then, install VG's dependencies. You'll need the protobuf and jansson development libraries installed, and to run the tests you will need:
* `jq`, `bc`, `rs`, and `parallel`
* `hexdump` and `column` from `bsdmainutils`
* [`npm` for testing documentation examples](https://github.com/anko/txm)).
* `jq`, `bc`, `rs`, and `parallel`
* `hexdump` and `column` from `bsdmainutils`
* [`npm` for testing documentation examples](https://github.com/anko/txm)).

On Ubuntu, you should be able to do:

make get-deps

If you get complaints that `sudo` is not found, install it:

apt update
apt install sudo

If you get a bunch of errors like `E: Unable to locate package build-essential`, make sure your package index files are up to date by running:

sudo apt update

On other distros, you will need to perform the equivalent of:
On other distros, or if you do not have root access, you will need to perform the equivalent of:

sudo apt-get install build-essential git cmake pkg-config libncurses-dev libbz2-dev \
protobuf-compiler libprotoc-dev libprotobuf-dev libjansson-dev \
automake gettext autopoint libtool jq bsdmainutils bc rs parallel \
npm curl unzip redland-utils librdf-dev bison flex gawk lzma-dev \
liblzma-dev liblz4-dev libffi-dev libcairo-dev libboost-all-dev \
libzstd-devel pybind11-dev python3-pybind11
libzstd-dev pybind11-dev python3-pybind11

Note that **Ubuntu 16.04** does not ship a sufficiently new Protobuf; vg requires **Protobuf 3** which will have to be manually installed.

Expand All @@ -85,22 +99,47 @@ Other libraries may be required. Please report any build difficulties.

Note that a 64-bit OS is required. Ubuntu 20.04 should work.

When you are ready, build with `. ./source_me.sh && make`, and run with `./bin/vg`.
#### Linux: Build

When you are ready, build with `. ./source_me.sh && make`. You can use `make -j16` to run 16 build threads at a time, which greatly accelerates the process. If you have more CPU cores, you can use higher numbers.

Note that vg can take anywhere from 10 minutes to more than an hour to compile depending on your machine and the number of threads used.

You can also produce a static binary with `make static`, assuming you have static versions of all the dependencies installed on your system.

#### Linux: Run

Once vg is built, the binary will be at `bin/vg` inside the vg repository directory. You can run it with:

```
./bin/vg
```

You can also add its directory to your `PATH` enviornment variable, so that you can invoke `vg` from any directory. To do that on Bash, use this command from the vg repository directory:

```
echo 'export PATH="${PATH}:'"$(pwd)"'/bin"' >>~/.bashrc
```

Then close your terminal and open a new one. Run `vg` to make sure it worked.

If it did not work, make sure that you have a `.bash_profile` file in your home directory that will run your `.bashrc`:
```
if [ -f ~/.bashrc ]; then
source ~/.bashrc
fi
```

### Building on MacOS

#### Clone VG
#### Mac: Clone VG

The first step is to clone the vg repository:

git clone --recursive https://github.com/vgteam/vg.git
cd vg

#### Install Dependencies
#### Mac: Install Dependencies

VG depends on a number of packages being installed on the system where it is being built. Dependencies can be installed using either [MacPorts](https://www.macports.org/install.php) or [Homebrew](http://brew.sh/).

Expand All @@ -118,17 +157,35 @@ Homebrew provides another package management solution for OSX, and may be prefer
# Install all the dependencies in the Brewfile
brew bundle

#### Build
#### Mac: Build

With dependencies installed, VG can now be built:

. ./source_me.sh && make

As with Linux, you can add `-j16` or other numbers at the end to run multiple build tasks at once, if your computer can handle them.

**Note that static binaries cannot yet be built for Mac.**

Our team has successfully built vg on Mac with GCC versions 4.9, 5.3, 6, 7, and 7.3, as well as Clang 9.0.

#### Migrating to ARM Macs
#### Mac: Run

Once vg is built, the binary will be at `bin/vg` inside the vg repository directory. You can run it with:

```
./bin/vg
```

You can also add its directory to your `PATH` enviornment variable, so that you can invoke `vg` from any directory. To do that on the default `zsh` Mac shell, use this command from the vg repository directory:

```
echo 'export PATH="${PATH}:'"$(pwd)"'/bin"' >>~/.zshrc
```

Then close your terminal and open a new one. Run `vg` to make sure it worked.

##### Migrate a VG installation from x86 to ARM

The Mac platform is moving to ARM, with Apple's M1, M1 Pro, M1 Max, and subsequent chip designs. The vg codebase supports ARM on Mac as well as on Linux. **The normal installation instructions work on a factory-fresh ARM Mac**.

Expand Down
10 changes: 6 additions & 4 deletions vgci/vgci.sh
Original file line number Diff line number Diff line change
Expand Up @@ -340,12 +340,14 @@ then

# Dependencies for running tests. Need numpy, scipy and sklearn for
# running toil-vg mapeval, and dateutils and reqests for
# ./mins_since_last_build.py. Make sure to get the giant buildable modules
# as binaries only to avoid wasting CI time building some slightly nicer
# version Pip prefers.
pip3 install pytest timeout_decorator requests dateutils
# ./mins_since_last_build.py. Need exactly the right version of requests
# for the Python Docker API to work (see
# <https://github.com/docker/docker-py/issues/3256>).
pip3 install pytest timeout_decorator 'requests==2.31.0' dateutils
# TODO: To upgrade to Numpy 1.24+, we need to remove usages of `numpy.int`
# AKA `np.int` from toil-vg. See <https://stackoverflow.com/a/74946903>
# Make sure to get the giant buildable modules as binaries only to avoid
# wasting CI time building some slightly nicer version Pip prefers.
pip3 install 'numpy==1.23.5' scipy scikit-learn --only-binary :all:

# Install Toil
Expand Down
Loading