Skip to content

Commit

Permalink
Version Bump - v1.1.0 (#39)
Browse files Browse the repository at this point in the history
* Version bump: v1.0.0 to v1.1.0
* README Updated: infrastructure examples removed and svg examples added
  • Loading branch information
bryan-bar authored Feb 28, 2023
1 parent 1b6ffdf commit 45bc090
Show file tree
Hide file tree
Showing 6 changed files with 35 additions and 283 deletions.
312 changes: 30 additions & 282 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,294 +23,20 @@ describing the target cloud infrastructure.
| Azure | CosmoDB | :x: |
| Azure | Azure Kubernetes Service |:white_check_mark:|

## Infrastructure file

Following are examples of infrastructure files describing the target cloud
infrastructure. Example yaml files found inside [infrastructure-examples directory](./infrastructure-examples/)

### AWS EC2 machines

```yaml
aws:
tags:
cluster_name: ec2-machines-demo
created_by: terraform
ssh_user: rocky
operating_system:
name: Rocky-8-ec2-8.6-20220515.0.x86_64
owner: 679593333241
regions:
us-east-1:
cidr_block: 10.0.0.0/16
zones:
us-east-1b: 10.0.0.0/24
us-east-1c: 10.0.1.0/24
service_ports:
- port: 22
protocol: tcp
description: "SSH"
- port: 30000
protocol: tcp
description: "DBT-2"
- port: 30000
protocol: udp
description: "DBT-2"
- port: 5432
protocol: tcp
description: "PostgreSQL"
machines:
dbt2-client:
type: dbt2-client
region: us-east-1
zone: us-east-1b
instance_type: c5.18xlarge
volume:
type: gp2
size_gb: 50
iops: 5000
encrypted: false
dbt2-driver:
type: dbt2-driver
region: us-east-1
zone: us-east-1b
instance_type: c5.18xlarge
volume:
type: gp2
size_gb: 50
iops: 5000
encrypted: false
pg1:
type: postgres
region: us-east-1
zone: us-east-1b
instance_type: c5.4xlarge
volume:
type: gp2
size_gb: 50
iops: 5000
encrypted: false
additional_volumes:
- mount_point: /opt/pg_data
size_gb: 200
type: io2
iops: 50000
encrypted: false
- mount_point: /opt/pg_wal
size_gb: 200
type: io2
iops: 50000
encrypted: false
```
### AWS RDS Database
```yaml
aws:
tags:
cluster_name: rds-database-demo
regions:
us-east-1:
cidr_block: 10.0.0.0/16
zones:
us-east-1a: 10.0.0.0/24
service_ports:
- port: 5432
protocol: tcp
description: "PostgreSQL"
databases:
mydb1:
region: us-east-1
engine: postgres
engine_version: 13
instance_type: db.t3.micro
dbname: "dbt2"
username: "postgres"
password: "12Password!"
port: 5432
volume:
size_gb: 100
type: io1
iops: 1000
encrypted: true
settings:
- name: checkpoint_timeout
value: 900
- name: max_connections
value: 300
- name: max_wal_size
value: 5000
- name: random_page_cost
value: 1.25
- name: work_mem
value: 16000
```
### AWS Aurora Database
```yaml
aws:
tags:
cluster_name: aurora-demo
regions:
us-east-1:
cidr_block: 10.0.0.0/16
zones:
us-east-1a: 10.0.1.0/24
us-east-1b: 10.0.2.0/24
service_ports:
- port: 5432
protocol: tcp
description: "PostgreSQL"
aurora:
mydb2:
region: us-east-1
zones:
- us-east-1a
- us-east-1b
count: 1
engine: aurora-postgresql
engine_version: 13
dbname: "test"
username: "postgres"
password: "12Password!"
port: 5432
instance_type: db.t3.medium
settings:
- name: max_connections
value: 300
- name: random_page_cost
value: 1.25
- name: work_mem
value: 16000
```
### AWS Buildbot Master and Worker
```yaml
aws:
tags:
cluster_name: BuildBot-Demo
ssh_user: ubuntu
operating_system:
name: ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-
owner: 099720109477
regions:
us-east-2:
cidr_block: 10.0.0.0/16
zones:
us-east-2b: 10.0.0.0/24
service_ports:
- port: 22
protocol: tcp
description: "SSH"
region_ports:
- port: 9989
protocol: tcp
description: "worker connection to master"
- port: 8010
protocol: tcp
description: "master web UI"
machines:
ebac-master:
count: 1
type: master
region: us-east-2
zone: us-east-2b
instance_type: c5.xlarge
volume:
type: gp2
size_gb: 50
iops: 5000
encrypted: false
ebac-worker-0:
type: worker
region: us-east-2
zone: us-east-2b
instance_type: c5.xlarge
volume:
type: gp2
size_gb: 50
iops: 5000
encrypted: false
additional_volumes:
- mount_point: /var/lib/buildbot-worker
size_gb: 300
type: io2
iops: 5000
encrypted: false
```
### GCloud Compute Engine VMs
```yaml
gcloud:
tags:
cluster_name: gcloud-infra
ssh_user: rocky
operating_system:
name: rocky-linux-8
regions:
us-west2:
cidr_block: 10.2.0.0/16
zones:
us-west2-b: 10.2.20.0/24
service_ports:
- port: 22
protocol: tcp
description: "SSH"
us-west1:
cidr_block: 10.1.0.0/16
zones:
us-west1-b: 10.1.20.0/24
us-west1-c: 10.1.30.0/24
service_ports:
- port: 22
protocol: tcp
description: "SSH"
region_ports:
- protocol: icmp
description: "ping"
machines:
dbt2-driver:
type: dbt2-driver
region: us-west2
zone: us-west2-b
instance_type: c2-standard-4
volume:
type: pd-standard
size_gb: 50
pg1:
type: postgres
region: us-west1
zone: us-west1-c
instance_type: e2-standard-4
service_ip: true
volume:
type: pd-standard
size_gb: 50
additional_volumes:
- mount_point: /opt/pg_data
type: pd-ssd
size_gb: 50
iops: null
- mount_point: /opt/pg_wal
type: pd-ssd
size_gb: 50
iops: null
```
#### Options:
* `service_ports`: ports open to the public
* `region_ports`: ports open and restricted to region's and cross-region's subnet cidrblocks

## Prerequisites and installation

The following components must be installed on the system:
- Python3
- Python3 >= 3.6
- AWS CLI
- GCloud CLI
- Azure CLI
- Terraform
- Terraform >= 1.3.6

## Infrastructure file examples

Infrastructure files describing the target cloud
can be found inside of the [infrastructure-examples directory](./infrastructure-examples/)


### Prequisites installation on Debian 11

Expand Down Expand Up @@ -387,6 +113,10 @@ resources creation:
$ edb-terraform ~/my_project -c aws my_infrastructure.yml --validate
```

<p align="center">
<img width="100%" src="./images/generate.svg">
</p>

2. Terraform initialisation of the *project*:
```shell
$ cd ~/my_project
Expand All @@ -399,6 +129,11 @@ resources creation:
$ terraform apply -auto-approve
```

<p align="center">
<img width="100%" src="./images/apply.svg">
</p>


## SSH access to the machines

Once cloud resources provisioning is completed, machines public and private IPs
Expand Down Expand Up @@ -427,11 +162,24 @@ servers:
[...]
```
You can also use `terraform output` to get a json object for use
```bash
terraform output -json servers | python3 -m json.tool
```
SSH key files: `ssh-id_rsa` and `ssh-id_rsa.pub`.
<p align="center">
<img width="100%" src="./images/ssh.svg">
</p>
## Cloud resources destruction
```shell
$ cd ~/my_project
$ terraform destroy -auto-approve
```
<p align="center">
<img width="100%" src="./images/destroy.svg">
</p>
2 changes: 1 addition & 1 deletion edbterraform/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "1.0.0"
__version__ = "1.1.0"
1 change: 1 addition & 0 deletions images/apply.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions images/destroy.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions images/generate.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions images/ssh.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 45bc090

Please sign in to comment.