Skip to content

Commit

Permalink
docs(ET-1813): add object storage configuration of reader
Browse files Browse the repository at this point in the history
  • Loading branch information
yidian1997 authored and gvasquezvargas committed Feb 6, 2025
1 parent 513c409 commit 0a839e3
Showing 1 changed file with 43 additions and 7 deletions.
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "Configuring and running the EDB DMS Reader"
deepToC: true
redirects:
redirects:
- /purl/dms/configure_source
---

Expand All @@ -19,7 +19,7 @@ redirects:

## Configuring the reader

1. Open the EDB DMS reader located in `/opt/cdcreader/run-cdcreader.sh` and ensure you have write permissions.
1. Open the EDB DMS reader located in `/opt/cdcreader/run-cdcreader.sh` and ensure you have write permissions.

1. Set the variables according to your environment and uncomment the edited lines. See [parameters](#parameters) for further guidance. The script is reproduced below.

Expand Down Expand Up @@ -71,7 +71,7 @@ redirects:
# The CATALOG is the database name
#export DBZ_DATABASES_0__CATALOG=source
#export DBZ_DATABASES_0__USERNAME=postgres
# The password env can be set without specifing it here
# The password env can be set without specifying it here
# but the env structure looks like this
#export DBZ_DATABASES_0__PASSWORD=password

Expand All @@ -87,6 +87,16 @@ redirects:
# but the env structure looks like this
#export DBZ_DATABASES_1__PASSWORD=password

##############################################################################
# DMS Reader Object storage config, only necessary for appliance #
##############################################################################
#export AWS_ACCESS_KEY_ID=
#export AWS_SECRET_ACCESS_KEY=
#export AWS_REGION=
#export AWS_CA_BUNDLE=
#export AWS_ENDPOINT_URL_S3=
#export BUCKET_NAME=

##########################################
# Optional Parameters Below #
##########################################
Expand All @@ -105,7 +115,7 @@ java ${JAVA_OPTS} -jar quarkus-run.jar

### `DBZ_ID`

This is the name you assign to identify a source. This name will later appear as a _source_ in the **Migrate** > **Sources** section of the EDB Postgres AI Console.
This is the name you assign to identify a source. This name will later appear as a _source_ in the **Migrate** > **Sources** section of the EDB Postgres AI Console.

Consider the following ID guidelines:

Expand All @@ -119,7 +129,7 @@ Specifies the URL of the service that will host the migration. `transporter-rw-s

### `TLS_PRIVATE_KEY_PATH`

Directory path to the `client-key.pem` private key you downloaded from the EDB Postgres AI Console.
Directory path to the `client-key.pem` private key you downloaded from the EDB Postgres AI Console.

The HTTP client of the EDB DMS Reader uses it to perform mTLS authentication with the `transporter-rw-service`.

Expand Down Expand Up @@ -148,9 +158,35 @@ Created from the Certificate Authority configured in `TLS_CA_PATH`.

The Apicurio client uses it to perform mTLS authentication with the `transporter-rw-service`.

### `AWS_ACCESS_KEY_ID`
1) If you are using AWS S3, You need to set the access key id with the read and write permission of the bucket.

2) If you are using s3 compatible storage like minio, you need to set the access key of your object storage.

### `AWS_SECRET_ACCESS_KEY`
1) If you are using AWS S3, You need to set the secret access key with the read and write permission of the bucket.

2) If you are using s3 compatible storage like minio, you need to set the secret key of your object storage.

### `AWS_REGION (Optional)`
1) If you are using AWS S3, You need to set this according to the region in AWS_ENDPOINT_URL_S3

2) If you are using s3 compatible storage like minio, you don't need to set this.

### `AWS_CA_BUNDLE (Optional)`
This is only needed if you are using a self-signed certificate for the object storage. You need to set the path to the CA bundle file.

### `AWS_ENDPOINT_URL_S3`
1) If you are using AWS S3, you can find the endpoint url in: https://docs.aws.amazon.com/general/latest/gr/s3.html

2) If you are using s3 compatible storage like minio, you need to set according to your object storage configuration.

### `BUCKET_NAME`
The bucket name you want to use for the migration. The bucket need to be created before the migration.

### `DBZ_DATABASES`

This is a list of source database information you require for the EDB DMS Reader be able to read the correct source database information for the migration.
This is a list of source database information you require for the EDB DMS Reader be able to read the correct source database information for the migration.

You can configure the EDB DMS Reader to migrate multiple databases. The `DBZ_DATABASES_0__TYPE` section delimits the information for the first database. You can use `DBZ_DATABASES_1__TYPE` to provide data for a second database. Add more sections to the EDB DMS Reader (`DBZ_DATABASES_2__TYPE`, `DBZ_DATABASES_3__TYPE`) by increasing the index manully.

Expand Down Expand Up @@ -181,7 +217,7 @@ The password for the database username of the source database.

## Running the EDB DMS Reader

1. Start the migration:
1. Start the migration:

```shell
cd /opt/cdcreader
Expand Down

0 comments on commit 0a839e3

Please sign in to comment.