diff --git a/README.md b/README.md index 55556f6f44d..f4714c8e6e6 100644 --- a/README.md +++ b/README.md @@ -55,27 +55,41 @@ The easiest way to get vg is to download one of our release builds for Linux. We If you don't want to or can't use a pre-built release of vg, or if you want to become a vg developer, you can build it from source instead. +#### Linux: Clone VG + First, obtain the repo and its submodules: git clone --recursive https://github.com/vgteam/vg.git cd vg + +#### Linux: Install Dependencies Then, install VG's dependencies. You'll need the protobuf and jansson development libraries installed, and to run the tests you will need: - * `jq`, `bc`, `rs`, and `parallel` - * `hexdump` and `column` from `bsdmainutils` - * [`npm` for testing documentation examples](https://github.com/anko/txm)). +* `jq`, `bc`, `rs`, and `parallel` +* `hexdump` and `column` from `bsdmainutils` +* [`npm` for testing documentation examples](https://github.com/anko/txm)). + On Ubuntu, you should be able to do: make get-deps + +If you get complaints that `sudo` is not found, install it: + + apt update + apt install sudo + +If you get a bunch of errors like `E: Unable to locate package build-essential`, make sure your package index files are up to date by running: + + sudo apt update -On other distros, you will need to perform the equivalent of: +On other distros, or if you do not have root access, you will need to perform the equivalent of: sudo apt-get install build-essential git cmake pkg-config libncurses-dev libbz2-dev \ protobuf-compiler libprotoc-dev libprotobuf-dev libjansson-dev \ automake gettext autopoint libtool jq bsdmainutils bc rs parallel \ npm curl unzip redland-utils librdf-dev bison flex gawk lzma-dev \ liblzma-dev liblz4-dev libffi-dev libcairo-dev libboost-all-dev \ - libzstd-devel pybind11-dev python3-pybind11 + libzstd-dev pybind11-dev python3-pybind11 Note that **Ubuntu 16.04** does not ship a sufficiently new Protobuf; vg requires **Protobuf 3** which will have to be manually installed. @@ -85,22 +99,47 @@ Other libraries may be required. Please report any build difficulties. Note that a 64-bit OS is required. Ubuntu 20.04 should work. -When you are ready, build with `. ./source_me.sh && make`, and run with `./bin/vg`. +#### Linux: Build + +When you are ready, build with `. ./source_me.sh && make`. You can use `make -j16` to run 16 build threads at a time, which greatly accelerates the process. If you have more CPU cores, you can use higher numbers. Note that vg can take anywhere from 10 minutes to more than an hour to compile depending on your machine and the number of threads used. You can also produce a static binary with `make static`, assuming you have static versions of all the dependencies installed on your system. +#### Linux: Run + +Once vg is built, the binary will be at `bin/vg` inside the vg repository directory. You can run it with: + +``` +./bin/vg +``` + +You can also add its directory to your `PATH` enviornment variable, so that you can invoke `vg` from any directory. To do that on Bash, use this command from the vg repository directory: + +``` +echo 'export PATH="${PATH}:'"$(pwd)"'/bin"' >>~/.bashrc +``` + +Then close your terminal and open a new one. Run `vg` to make sure it worked. + +If it did not work, make sure that you have a `.bash_profile` file in your home directory that will run your `.bashrc`: +``` +if [ -f ~/.bashrc ]; then + source ~/.bashrc +fi +``` + ### Building on MacOS -#### Clone VG +#### Mac: Clone VG The first step is to clone the vg repository: git clone --recursive https://github.com/vgteam/vg.git cd vg -#### Install Dependencies +#### Mac: Install Dependencies VG depends on a number of packages being installed on the system where it is being built. Dependencies can be installed using either [MacPorts](https://www.macports.org/install.php) or [Homebrew](http://brew.sh/). @@ -118,17 +157,35 @@ Homebrew provides another package management solution for OSX, and may be prefer # Install all the dependencies in the Brewfile brew bundle -#### Build +#### Mac: Build With dependencies installed, VG can now be built: . ./source_me.sh && make + +As with Linux, you can add `-j16` or other numbers at the end to run multiple build tasks at once, if your computer can handle them. **Note that static binaries cannot yet be built for Mac.** Our team has successfully built vg on Mac with GCC versions 4.9, 5.3, 6, 7, and 7.3, as well as Clang 9.0. -#### Migrating to ARM Macs +#### Mac: Run + +Once vg is built, the binary will be at `bin/vg` inside the vg repository directory. You can run it with: + +``` +./bin/vg +``` + +You can also add its directory to your `PATH` enviornment variable, so that you can invoke `vg` from any directory. To do that on the default `zsh` Mac shell, use this command from the vg repository directory: + +``` +echo 'export PATH="${PATH}:'"$(pwd)"'/bin"' >>~/.zshrc +``` + +Then close your terminal and open a new one. Run `vg` to make sure it worked. + +##### Migrate a VG installation from x86 to ARM The Mac platform is moving to ARM, with Apple's M1, M1 Pro, M1 Max, and subsequent chip designs. The vg codebase supports ARM on Mac as well as on Linux. **The normal installation instructions work on a factory-fresh ARM Mac**. diff --git a/vgci/vgci.sh b/vgci/vgci.sh index 192d6d19839..54b52f09911 100755 --- a/vgci/vgci.sh +++ b/vgci/vgci.sh @@ -340,12 +340,14 @@ then # Dependencies for running tests. Need numpy, scipy and sklearn for # running toil-vg mapeval, and dateutils and reqests for - # ./mins_since_last_build.py. Make sure to get the giant buildable modules - # as binaries only to avoid wasting CI time building some slightly nicer - # version Pip prefers. - pip3 install pytest timeout_decorator requests dateutils + # ./mins_since_last_build.py. Need exactly the right version of requests + # for the Python Docker API to work (see + # ). + pip3 install pytest timeout_decorator 'requests==2.31.0' dateutils # TODO: To upgrade to Numpy 1.24+, we need to remove usages of `numpy.int` # AKA `np.int` from toil-vg. See + # Make sure to get the giant buildable modules as binaries only to avoid + # wasting CI time building some slightly nicer version Pip prefers. pip3 install 'numpy==1.23.5' scipy scikit-learn --only-binary :all: # Install Toil