Skip to content

Commit

Permalink
Merge PR #56: Sommelier end-to-end rebalance test WIP
Browse files Browse the repository at this point in the history
* modifying oracle

* removing oracle specific types

* serializing tick weights

* mostly removed oracle

* validation of commits

* weighted tick map

* makefile updates

* multiple

* oracle feeder doesnt live here

* consistent naming

* no more oracle

* add proto defs

* cleanup

* clean up params

* WIP outgoing_tx types

* remove miss counter

* new key style

* undelete

* rename to gravity

* initial integration test setup

* codec files unneeded

* codec data types not needed

* no miss counters or data types

* cleanup

* gofmt

* linting

* go fmt

* goimports

* go mod tidy

* remove empty comments

* disable critic and interfacer

* disable whiny linters

* no args

* prealloc

* call notes

* custom governance

* keeper updates

* split proposals

* sdk updates

* more sdk changes

* go mod

* cellar updates

* commit cellar

* Comment out allocation module from app.go to prepare for merge

* governance implementation

* fmt

* go imports

* no implicit mem addr

* integration tests

* linting

* abi updates

* WIP rebalance call

* test fix

* Merge PR #60: Rebalance Call test

* Passing pack teset

* Add comments

* Fix string -> bytes

* Check error

* initial keeper tests

* eth key from mnemonic

* file rename

* mnemonics everywhere

* fmt

* imports

* Integration test hardhat (#64)

* setup hardhat folder

* setup node test

* clean up task cruft

* pin block numer

* formatted

* dockerfile for hardhat

* makefile

* cleanup

* initial genesis

* genesis file

* nodes talking, can't be reached by rpc

* validators operating

* fix docker setup for hardhat (#68)

* fix docker setup for hardhat

* check in artifacts

* cleanup

* bump orc version to gorc image

* gorc running

* ethereum startup check

* make command to run test

* ci integration tests

* split out version

* incorrect depgs

* formatting

* no version

* build context

* wrong dir

* rename to proper name

* proper path

* refactor common code

* path instead of passphrase

* typo

* key paths

* fund accounts with eth

* wait for balance

* running chain

* cellars in genesis state

* signing and sending msgs

* failing on from retrieval

* sending error

* registered auth codec

* longer test timeout

* proper format

* update eth version

* unknown error

* check ticks after rebalance

* more logs in startup of validators

* somm not val

* quick exit on exit

* fail faster

* get status

* use gravity delegations instead of custom

* different signer

* back to the validator

* use creator

* use working dir in CI

* getting past precommits

* cellars not saving

* precommits successful

* wait for new vote period

* hash mismatch

* hash logging

* consolidate caladdr logic

* new datahash

* waiting for end

* calling contract

* signer set not updating

* more logging

* logging args

* point at pr orchestrator

* more logs

* update for more logs

* shed somm user

* empty line

* Fix genesis tests.

Show hashes in hex form

* linting

* linting pass

* match the gravity_id in genesis with hardhard id

* trying to inject contract source

* update oz folder structure, sol compiler vers

* contracts

* update block #, gasPrice

* lock contracts to previous version

* runs

* checkin artifacts

* verification

* verify is no help

* cleanup

* redelegate

* wrong denom

Co-authored-by: Jack Zampolin <[email protected]>
Co-authored-by: Kevin Kennis <[email protected]>
Co-authored-by: Zaki Manian <[email protected]>
Co-authored-by: Lucky Odisetti <[email protected]>
  • Loading branch information
5 people authored Oct 18, 2021
1 parent 6df8023 commit 722f0a5
Show file tree
Hide file tree
Showing 113 changed files with 110,666 additions and 1,963 deletions.
7 changes: 7 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
integration_tests/
!integration_tests/gorc_bootstrap.sh
docs/
Dockerfile

.idea/
.github/
163 changes: 163 additions & 0 deletions .github/workflows/integration-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
name: Integration tests

on:
push:
branches:
- main
tags:
- 'v*.*.*'
pull_request:

env:
REGISTRY: ghcr.io
IMAGE_NAME: ${{ github.repository }}
ORCHESTRATOR_IMAGE_NAME: ghcr.io/peggyjv/gravity-bridge-orchestrator:v0.2.1

jobs:
sommelier-build:
permissions:
contents: read
packages: write
runs-on: ubuntu-latest
steps:
- name: checkout
uses: actions/checkout@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Cache Docker layers
uses: actions/cache@v2
id: docker-cache
with:
path: /tmp/.buildx-cache
# Key is named differently to avoid collision
key: ${{ runner.os }}-multi-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-multi-buildx
- name: container-login
uses: docker/login-action@v1
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: labels
id: meta
uses: docker/metadata-action@v3
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-sommelier
- name: build-and-push
uses: docker/build-push-action@v2
with:
context: .
file: Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
builder: ${{ steps.buildx.outputs.name }}
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,mode=max,dest=/tmp/.buildx-cache-new
- name: Move Docker cache
run: |
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
hardhat-build:
permissions:
contents: read
packages: write
runs-on: ubuntu-latest
steps:
- name: checkout
uses: actions/checkout@v2
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v1
- name: Cache Docker layers
uses: actions/cache@v2
id: docker-cache
with:
path: /tmp/.buildx-cache
# Key is named differently to avoid collision
key: ${{ runner.os }}-multi-buildx-${{ github.sha }}
restore-keys: |
${{ runner.os }}-multi-buildx
- name: container-login
uses: docker/login-action@v1
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: labels
id: meta
uses: docker/metadata-action@v3
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-hardhat
- name: build-and-push
uses: docker/build-push-action@v2
with:
context: integration_tests/ethereum
file: integration_tests/ethereum/Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
builder: ${{ steps.buildx.outputs.name }}
cache-from: type=local,src=/tmp/.buildx-cache
cache-to: type=local,mode=max,dest=/tmp/.buildx-cache-new
- name: Move Docker cache
run: |
rm -rf /tmp/.buildx-cache
mv /tmp/.buildx-cache-new /tmp/.buildx-cache
integration-tests:
runs-on: ubuntu-latest
needs: [ sommelier-build, hardhat-build ]
strategy:
fail-fast: false
matrix:
test_type: [ "Rebalance" ]
steps:
- name: Set up Go 1.16
uses: actions/setup-go@v2
with:
go-version: ^1.16
- name: checkout
uses: actions/checkout@v2
- name: go-cache
uses: actions/cache@v2
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
- name: container-login
uses: docker/login-action@v1
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: docker sommelier metadata
id: meta-sommelier
uses: docker/metadata-action@v3
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-sommelier
- name: docker hardhat metadata
id: meta-hardhat
uses: docker/metadata-action@v3
with:
images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}-hardhat
- name: pull sommelier image
run: docker pull ${{ steps.meta-sommelier.outputs.tags }}
- name: rename sommelier image
run: docker tag ${{ steps.meta-sommelier.outputs.tags }} sommelier:prebuilt
- name: pull hardhat image
run: docker pull ${{ steps.meta-hardhat.outputs.tags }}
- name: rename hardhat image
run: docker tag ${{ steps.meta-hardhat.outputs.tags }} ethereum:prebuilt
- name: pull orchestrator image
run: docker pull ${{ env.ORCHESTRATOR_IMAGE_NAME }}
- name: rename orchestrator image
run: docker tag ${{ env.ORCHESTRATOR_IMAGE_NAME }} orchestrator:prebuilt
- name: build go test binary
run: cd integration_tests && go test -c
- name: Run end to end test
run: ./integration_tests/integration_tests.test -test.failfast -test.v -test.run IntegrationTestSuite -testify.m Test${{ matrix.test_type }}
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -62,4 +62,5 @@ node_modules
solidity/dist
cache
artifacts
!integration_tests/ethereum/artifacts
*.test
6 changes: 4 additions & 2 deletions .golangci.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
run:
tests: false
#run:
# tests: false
# # timeout for analysis, e.g. 30s, 5m, default is 1m
# timeout: 5m

Expand Down Expand Up @@ -59,6 +59,8 @@ issues:
max-issues-per-linter: 10000
max-same-issues: 10000

fix: true

linters-settings:
dogsled:
max-blank-identifiers: 3
Expand Down
34 changes: 22 additions & 12 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,34 +1,44 @@
FROM golang:alpine AS build-env

# Set up dependencies
ENV PACKAGES curl make git libc-dev bash gcc linux-headers eudev-dev python3
RUN apk add --no-cache curl make git libc-dev bash gcc linux-headers eudev-dev python3

# Set working directory for the build
WORKDIR /go/src/github.com/peggyjv/sommelier

# Get dependancies - will also be cached if we won't change mod/sum
COPY go.mod .
COPY go.sum .
RUN go mod download

# Add source files
COPY . .

# Install minimum necessary dependencies, build Cosmos SDK, remove packages
RUN apk add --no-cache $PACKAGES && \
make install
# build Sommelier
RUN make install

# Final image
FROM alpine:edge

ENV SOMM /somm
#ENV SOMM /somm

# Install ca-certificates
RUN apk add --update ca-certificates
RUN apk add --update ca-certificates bash

# create and expose config dir
#RUN mkdir -p /somm/.sommelier/config
#RUN chmod -R 777 /somm/.sommelier/config
#RUN mkdir /somm/.sommelier/data
#RUN chmod 777 /somm/.sommelier/data

RUN addgroup sommuser && \
adduser -S -G sommuser sommuser -h "$SOMM"
# RUN addgroup sommuser && \
# adduser -S -G sommuser sommuser -h "$SOMM"

USER sommuser
#USER sommuser
#EXPOSE 1317 6060 6061 6062 6063 6064 6065 9090 26656 26657

WORKDIR $SOMM
#WORKDIR $SOMM

# Copy over binaries from the build-env
COPY --from=build-env /go/bin/sommelier /usr/bin/sommelier

CMD ["sommelier"]
CMD ["sommelier", "start"]
52 changes: 52 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -346,3 +346,55 @@ tools-clean:
rm -f proto-tools-stamp buf-stampmodule

.PHONY: proto-all proto-gen proto-swagger-gen proto-format proto-lint proto-check-breaking proto-update-deps

#####################
# Integration tests #
#####################

#ORCHESTRATOR_IMAGE := "ghcr.io/peggyjv/gravity-bridge-orchestrator:v0.2.13"
# pointing at a specific PR currently, for testing
ORCHESTRATOR_IMAGE := "ghcr.io/peggyjv/gravity-bridge-orchestrator:pr-203"

e2e_build_images:
@docker pull $(ORCHESTRATOR_IMAGE)
@docker tag $(ORCHESTRATOR_IMAGE) orchestrator:prebuilt
@docker build -t sommelier:prebuilt -f Dockerfile .
@docker build -t ethereum:prebuilt -f integration_tests/ethereum/Dockerfile integration_tests/ethereum/

e2e_clean_slate:
@docker rm --force \
$(shell docker ps -qa --filter="name=ethereum") \
$(shell docker ps -qa --filter="name=sommelier") \
$(shell docker ps -qa --filter="name=orchestrator") \
1>/dev/null \
2>/dev/null \
|| true
@docker wait \
$(shell docker ps -qa --filter="name=ethereum") \
$(shell docker ps -qa --filter="name=sommelier") \
$(shell docker ps -qa --filter="name=orchestrator") \
1>/dev/null \
2>/dev/null \
|| true
@docker network prune --force 1>/dev/null 2>/dev/null || true
@cd integration_tests && go test -c

e2e_basic: e2e_clean_slate
@E2E_SKIP_CLEANUP=true
@integration_tests/integration_tests.test -test.run TestBasicChain -test.failfast -test.v || make -s fail

e2e_rebalance: e2e_clean_slate
@E2E_SKIP_CLEANUP=true integration_tests/integration_tests.test -test.failfast -test.v -test.run IntegrationTestSuite -testify.m TestRebalance || make -s fail

fail:
@echo 'test failed; dumping container logs into ./testlogs for review'
@docker logs ethereum > testlogs/ethereum.log>&1 || true
@docker logs sommelier0 > testlogs/sommelier0.log 2>&1 || true
@docker logs sommelier1 > testlogs/sommelier1.log 2>&1 || true
@docker logs sommelier2 > testlogs/sommelier2.log 2>&1 || true
@docker logs sommelier3 > testlogs/sommelier3.log 2>&1 || true
@docker logs orchestrator0 > testlogs/orchestrator0.log 2>&1 || true
@docker logs orchestrator1 > testlogs/orchestrator1.log 2>&1 || true
@docker logs orchestrator2 > testlogs/orchestrator2.log 2>&1 || true
@docker logs orchestrator3 > testlogs/orchestrator3.log 2>&1 || true
@false
Loading

0 comments on commit 722f0a5

Please sign in to comment.