Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bring local-setup into the V2 era #156

Closed
wants to merge 15 commits into from
Closed

Bring local-setup into the V2 era #156

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
88214ca
Clear out old v1 cruft
Jun 22, 2023
87744d6
Encapsulate bitcoin docker stuff
Jun 22, 2023
4e72837
Add the v2 submodules
Jun 22, 2023
e4d92ed
Add geth genesis data
Jun 22, 2023
678c622
Add a geth docker service
Jun 22, 2023
2e83f60
Add a geth reset script
Jun 22, 2023
32520a3
Bundle everything into docker-compose
Jun 22, 2023
ab7a71d
Include the data generation folders in .gitignore
Jun 22, 2023
c1569d7
Bitcoind doesn't need jq
Jun 26, 2023
aede19d
Add a reference to the transitive git problem
beaurancourt Jul 12, 2023
b195086
Put the submodules in parent directories
beaurancourt Jul 12, 2023
facc694
Remove `git init` from the build steps
beaurancourt Jul 12, 2023
c42e76a
Comment the mysterious yarn build || true
beaurancourt Jul 12, 2023
1a46f7c
Comment the mysterioud commit checkout in electrs
beaurancourt Jul 12, 2023
8c71a0e
Nuke the workflows
beaurancourt Jul 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .dockerignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
**/.git
**/node_modules
5 changes: 3 additions & 2 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
.idea
/ethereum/.data/
/bitcoin/.data/
/docker/bitcoin/.data/
/docker/ethereum/.data/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should hold ethereum explorer cache, right? I'm asking because I don't have this folder, despite successfully running the docker container (likely related to seeing b-explorer | Error trying to update latest Transactions cache. in the container logs).

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The transaction cache stuff is expected - i'll call that out in the PR description

./docker/ethereum/.data/ is where we're storing ethereum chain data. docker-compose.yml is handling the volume work for us:

  geth:
    container_name: geth
    build:
      context: .
      dockerfile: Dockerfile.geth
    volumes:
      - ./docker/geth/.data:/ethereum/data

/docker/geth/.data/
/storage/
node_modules
vagrant-box
Expand Down
28 changes: 8 additions & 20 deletions .gitmodules
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,21 +1,9 @@
[submodule "keep-core"]
path = keep-core
url = https://github.com/keep-network/keep-core.git
[submodule "keep-ecdsa"]
path = keep-ecdsa
url = https://github.com/keep-network/keep-ecdsa.git
[submodule "tbtc"]
path = tbtc
url = https://github.com/keep-network/tbtc.git
[submodule "tbtc-dapp"]
path = tbtc-dapp
url = https://github.com/keep-network/tbtc-dapp.git
[submodule "tbtc.js"]
path = tbtc.js
url = https://github.com/keep-network/tbtc.js.git
[submodule "relays"]
path = relays
url = https://github.com/keep-network/relays.git
[submodule "coverage-pools"]
path = coverage-pools
url = https://github.com/keep-network/coverage-pools.git
path = keep-core
url = https://github.com/keep-network/keep-core.git
[submodule "solidity-contracts"]
path = solidity-contracts
url = https://github.com/threshold-network/solidity-contracts
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not in this PR but we could think about introducing some structure for submodules, for example:

submodules
  keep-network
    keep-core
    tbtc-v2
  threshold-network
    solidity-contracts

With the current setup, the solidity-contracts directory is confusing without looking at .gitmodules. What solidity contracts are we talking about? Are those tBTC or Keep contracts?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

went ahead and did it b195086

[submodule "tbtc-v2"]
path = tbtc-v2
url = https://github.com/keep-network/tbtc-v2
71 changes: 71 additions & 0 deletions Dockerfile.geth
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
ARG NODE_VERSION=16.14.0
ARG ALPINE_VERSION=3.16
ARG PYTHON_VERSION=3.10.4

FROM node:${NODE_VERSION}-alpine AS node

FROM python:${PYTHON_VERSION}-alpine${ALPINE_VERSION}

COPY --from=node /usr/lib /usr/lib
COPY --from=node /usr/local/share /usr/local/share
COPY --from=node /usr/local/lib /usr/local/lib
COPY --from=node /usr/local/include /usr/local/include
COPY --from=node /usr/local/bin /usr/local/bin

ENV GETH_DATA_DIR=/ethereum/data/
ENV GETH_ETHEREUM_ACCOUNT=0x944466f81e70c74c0f6b736ba72c7a69e475735a
ENV KEEP_ETHEREUM_PASSWORD=password
ENV NETWORK=development

RUN apk update
RUN apk add --update --no-cache bash git geth make gcc g++ && \
npm install --global yarn --force

# There's a transitive git problem that this fixes
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it this? Could be adding some reference in case the root cause is resolved in the future.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah!

aede19d

RUN git config --global url."https://".insteadOf git://

COPY ./solidity-contracts /solidity-contracts
WORKDIR /solidity-contracts
RUN git init
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need togit init? I think we should be able to remove it from here, as well as line 37 and 57.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i forget which repository needs to register as a git repo for the later commands to work, but it's at least one of them

having the projects as submodules means that rather than having a .git/ folder, they have .git files that point back at the parent directory, and that parent directory doesn't exist when we're copying

I can definitely go back and see which ones we can safely remove!

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh! i think the git init thing was required when i was actually running keep-core's install.sh. Since I'm not anymore (it's unpacked into the docker file and entrypoint), I can remove all of them 🎉

good catch! facc694

RUN yarn install
RUN yarn build
RUN yarn link
RUN yarn prepack

COPY ./keep-core /keep-core
WORKDIR /keep-core
RUN git init

WORKDIR /keep-core/solidity/random-beacon
RUN yarn install
RUN yarn link @threshold-network/solidity-contracts
RUN yarn build
RUN yarn link
RUN yarn prepack

WORKDIR /keep-core/solidity/ecdsa
RUN rm -rf .openzeppelin/unknown-*.json
RUN yarn link @threshold-network/solidity-contracts
RUN yarn link @keep-network/random-beacon
RUN yarn install
RUN yarn build
RUN yarn link
RUN yarn prepack

COPY ./tbtc-v2 /tbtc-v2
WORKDIR /tbtc-v2
RUN git init
WORKDIR /tbtc-v2/solidity
RUN yarn install --network-timeout 1000000 && \
yarn build || true
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why || true? Should we continue if yarn build failed?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add a comment, but for whatever reason the first yarn build almost always fails and the second yarn build always succeeds. happened both in docker and locally for me

rather than go down that rabbit hole, i just have docker try to build it twice

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment: c42e76a

RUN yarn link @threshold-network/solidity-contracts
RUN yarn link @keep-network/random-beacon
RUN yarn link @keep-network/ecdsa
RUN yarn build
RUN yarn prepack

WORKDIR /

ADD ./docker/geth/entrypoint.sh .

CMD /entrypoint.sh
15 changes: 0 additions & 15 deletions bitcoin/bitcoind/Dockerfile
View file
Open in desktop

This file was deleted.

58 changes: 0 additions & 58 deletions bitcoin/docker-compose.yml
View file
Open in desktop

This file was deleted.

31 changes: 0 additions & 31 deletions bitcoin/electrumx/Dockerfile
View file
Open in desktop

This file was deleted.

7 changes: 0 additions & 7 deletions bitcoin/electrumx/bin/init
View file
Open in desktop

This file was deleted.

6 changes: 0 additions & 6 deletions common.js/.eslintrc
View file
Open in desktop

This file was deleted.

3 changes: 0 additions & 3 deletions common.js/README.md
View file
Open in desktop

This file was deleted.

28 changes: 0 additions & 28 deletions common.js/package.json
View file
Open in desktop

This file was deleted.

64 changes: 0 additions & 64 deletions common.js/src/contracts.js
View file
Open in desktop

This file was deleted.

3 changes: 0 additions & 3 deletions common.js/src/index.js
View file
Open in desktop

This file was deleted.

Loading