Skip to content

Commit

Permalink
[build] Allow gitian-build.py to run on macOS
Browse files Browse the repository at this point in the history
Co-authored-by: Andrew#128 <[email protected]>

Summary
-------

As a follow-up to !1248, it turns out with some minor modifications,
it's possible to build using gitian on Docker for macOS!  This may save
time for people that run macOS natively (such as me).

In short: This MR allows the `--docker` options to work on macOS without failing.

In a future MR I will also compose some documentation about how to get
it working on macOS 100%.

For now it also needs you to use a fork of gitian-builder from here:

    https://github.com/cculianu/gitian-builder.git
    BRANCH: macos_support_no_debian_cache

There is a PR for this change that I submitted to the main
gitian-builder repo: devrandom/gitian-builder#252

Test Plan
---------

- Run gitian builder on Linux as normal to ensure that nothing broke.

**Optional:**

- If you are on a **macOS** host, get docker installed and running, check-out
  this branch, and also check-out: https://github.com/cculianu/gitian-builder.git
  branch: `macos_support_no_debian_cache` into your `gitian-builder`
  directory.
- Put the modified script from this MR on the top-level, next to your
  `bitcoin-cash-node` directory.
- Run: `./gitian-build.py --setup --docker`
- Run: `./gitian-build.py --docker -b -D -n -c -o lwm "satoshi" master`
- After a great deal of time waiting for it to rebuild the universe, you
  should have some static binaries built!
  • Loading branch information
cculianu authored and ftrader committed Nov 19, 2021
1 parent f4c6d7c commit 1789e3f
Show file tree
Hide file tree
Showing 3 changed files with 44 additions and 29 deletions.
41 changes: 25 additions & 16 deletions contrib/gitian-build.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,25 +15,34 @@
BOLD = ('\033[0m', '\033[1m')
RED = ('\033[0m', '\033[0;31m')

is_mac = sys.platform == 'darwin'


def setup():
global args, workdir
programs = ['ruby', 'git', 'apt-cacher-ng', 'make', 'wget']
if args.kvm:
programs += ['python-vm-builder', 'qemu-kvm', 'qemu-utils']
elif args.docker:
dockers = ['docker.io', 'docker-ce']
for i in dockers:
return_code = subprocess.call(
['sudo', 'apt-get', 'install', '-qq', i])
if return_code == 0:
break
if return_code != 0:
print('Cannot find any way to install docker', file=sys.stderr)
exit(1)
if not is_mac:
programs = ['ruby', 'git', 'apt-cacher-ng', 'make', 'wget']
if args.kvm:
programs += ['python-vm-builder', 'qemu-kvm', 'qemu-utils']
elif args.docker:
dockers = ['docker.io', 'docker-ce']
for i in dockers:
return_code = subprocess.call(
['sudo', 'apt-get', 'install', '-qq', i])
if return_code == 0:
break
if return_code != 0:
print('Cannot find any way to install docker', file=sys.stderr)
exit(1)
else:
programs += ['lxc', 'debootstrap']
subprocess.check_call(['sudo', 'apt-get', 'install', '-qq'] + programs)
else:
programs += ['lxc', 'debootstrap']
subprocess.check_call(['sudo', 'apt-get', 'install', '-qq'] + programs)
if args.docker:
print("Warning: macOS support is experimental and requires docker be already installed", file=sys.stderr)
else:
sys.exit("macOS support only works in --docker mode")

if not os.path.isdir('gitian-builder'):
subprocess.check_call(
['git', 'clone', 'https://github.com/devrandom/gitian-builder.git'])
Expand Down Expand Up @@ -240,7 +249,7 @@ def main():
args = parser.parse_args()
workdir = os.getcwd()

args.is_bionic = b'bionic' in subprocess.check_output(
args.is_bionic = not is_mac and b'bionic' in subprocess.check_output(
['lsb_release', '-cs'])

if args.kvm and args.docker:
Expand Down
6 changes: 3 additions & 3 deletions doc/gitian-building.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,9 @@ More independent Gitian builders are needed, which is why this guide exists.
It is preferred you follow these steps yourself instead of using someone else's
VM image to avoid 'contaminating' the build.

If you are running Debian or Ubuntu, see
[Running Gitian with Docker on Ubuntu/Debian](./gitian-building/gitian-building-docker.md)
for a relitively straightforward setup and build process.
If you are running Debian, Ubuntu or Mac, see
[Running Gitian with Docker](./gitian-building/gitian-building-docker.md)
for a relatively straightforward setup and build process.

The instructions below use the automated script [gitian-build.py](../contrib/gitian-build.py)
which only works in Debian/Ubuntu. For manual steps and instructions for fully
Expand Down
26 changes: 16 additions & 10 deletions doc/gitian-building/gitian-building-docker.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,34 @@
# Running Gitian with Docker on Ubuntu/Debian
This is a streamlined guide for running Gitian builds with Docker on Ubuntu or
Debian hardware. The following steps have been tested on Ubuntu 18.04.05.
Similar, if not the same steps should work on other versions of Ubuntu and
Debian.
# Running Gitian with Docker
This is a streamlined guide for running Gitian builds with Docker on Ubuntu,
Debian or Mac hardware.

# Setup
Ensure you have Docker installed. See https://docs.docker.com/get-docker/ for
installation instructions.

Make sure to go through the Linux post-install walkthrough, especially the
If you're using Linux, make sure to go through the Linux post-install
walkthrough, especially the
[Manage Docker as a non-root user](https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user)
section, to avoid having to use `sudo` all the time.

Prepare a workspace directory (e.g. `~/bchn-gitian`) and `cd` into it. You'll
## Prepare a build workspace
Create a workspace directory (e.g. `~/bchn-gitian`) and `cd` into it. You'll
only need to run through this setup once so long as you retain the workspace.

```bash
# Install dependencies
sudo apt install curl git
## Install dependencies
You'll need `git` and `curl` installed. If on Linux, just do `sudo apt install
curl git`.

## Gitian setup
```bash
# Fetch the `gitian-build.py` script
git clone https://gitlab.com/bitcoin-cash-node/bitcoin-cash-node.git
cp bitcoin-cash-node/contrib/gitian-build.py .

# If you are on a MacOS host, you will need the MacOS-capable fork
# of gitian-builder, if on Linux, you can skip this step.
git clone -b macos_support_no_debian_cache https://github.com/cculianu/gitian-builder.git

# Run the initial Gitian setup
./gitian-build.py --docker --setup

Expand Down

0 comments on commit 1789e3f

Please sign in to comment.