Add Default Ingestion Configuration Logging and Update Documentation …

…to Reflect Runtime Naming Convention (#49) * Change all references for pre-compiled binaries in the documentation to use the bootstrap naming convention. Add info logging about which database, table, and region are configured for the Prometheus Connector. Signed-off-by: forestmvey <[email protected]> * Change configuration default logs to debug level, and added log for lambda invocation. Signed-off-by: forestmvey <[email protected]> * Updating logs to level info for ingestion configurations. Signed-off-by: forestmvey <[email protected]> * Fix typo and grammar. Signed-off-by: forestmvey <[email protected]> * Updated log for query client in lambda handler. Signed-off-by: forestmvey <[email protected]> * Added check when running binary that default database and table values are set. Signed-off-by: forestmvey <[email protected]> * Revising initialization log format. Signed-off-by: forestmvey <[email protected]> --------- Signed-off-by: forestmvey <[email protected]>
awslabs · Mar 27, 2024 · 0565dac · 0565dac
1 parent 0f31097
commit 0565dac
Show file tree

Hide file tree

Showing 2 changed files with 27 additions and 22 deletions.
diff --git a/README.md b/README.md
@@ -133,20 +133,16 @@ This is the easiest and recommended method for running the connector.
 
 ### Run with Precompiled Binaries
 
-Run the precompiled binaries with required arguments `default-database` and `default-table`.
+The pre-compiled binaries independent of platform will have the name `bootstrap` to align with the `provided.al2023` lambda runtime naming convention. Run the precompiled binaries with required arguments `default-database` and `default-table`.
 
-| Platform | Command                                                      |
-| -------- | ------------------------------------------------------------ |
-| Linux    | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=prometheusDatabase  --default-table=prometheusMetricsTable` |
-| macOS    | `./timestream-prometheus-connector-darwin-amd64-1.0.0 --default-database=prometheusDatabase  --default-table=prometheusMetricsTable` |
-| Windows  | `timestream-prometheus-connector-windows-amd64-1.0.0 --default-database=prometheusDatabase  --default-table=prometheusMetricsTable` |
+`./bootstrap --default-database=prometheusDatabase  --default-table=prometheusMetricsTable`
 
 It is recommended to secure the Prometheus requests with TLS encryption. To enable TLS encryption:
 
 1. Specify the server certificate and the server private key through the `tls-certificate` and `tls-key` configuration options. An example for macOS is as follows:
 
    ```shell
-   ./timestream-prometheus-connector-darwin-amd64-1.0.0 \
+   ./bootstrap \
    --default-database=prometheusDatabase \
    --default-table=prometheusMetricsTable \
    --tls-certificate=serverCertificate.crt \
@@ -163,7 +159,7 @@ For more examples on configuring the Prometheus Connector see [Configuration Opt
 
 The following error message may show up when running the precompiled binary on macOS:
 
-`"timestream-prometheus-connector-darwin-amd64" cannot be opened because the developer cannot be verified.`
+`"bootstrap" cannot be opened because the developer cannot be verified.`
 
 Follow these steps to resolve:
 
@@ -391,7 +387,7 @@ To provide access to this newly created role, add a permission to the current us
 Go to [Configuration Options](#configuration-options) to see more information.
 9. Scroll down to basic settings.
 10. Click `Edit`.
-11. In the `Handler` section, enter the name of the Amazon Timestream Prometheus Connector ZIP file, which will be `timestream-prometheus-connector-1.0.0`.
+11. In the `Handler` section, enter the name of the Amazon Timestream Prometheus Connector ZIP file, which will be `bootstrap`.
 12. Click `Save`.
 
 #### Create the API on Amazon API Gateway
@@ -544,28 +540,28 @@ The default-database name and default-table name are required for data ingestion
 
    | Runtime              | Command                                                                                                                                                                                             |
    | -------------------- |-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-   | Precompiled Binaries | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=PrometheusDatabase  --default-table=PrometheusMetricsTable --region=us-west-2`                                              |
+   | Precompiled Binaries | `./bootstrap --default-database=PrometheusDatabase  --default-table=PrometheusMetricsTable --region=us-west-2`                                              |
    | AWS Lambda Function  | `aws lambda update-function-configuration --function-name PrometheusConnector --environment "Variables={default_database=prometheusDatabase,default_table=prometheusMetricsTable,region=us-west-2}"` |
 
 2. Configure the Prometheus Connector listen for requests on an HTTPS server `https://localhost:9201` with TLS encryption.
 
    | Runtime              | Command                                                                                                                                                                                                    |
    | -------------------- |------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-   | Precompiled Binaries | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=PrometheusDatabase  --default-table=PrometheusMetricsTable --tls-certificate=serverCertificate.crt --tls-key=serverPrivateKey.key` |
+   | Precompiled Binaries | `./bootstrap --default-database=PrometheusDatabase  --default-table=PrometheusMetricsTable --tls-certificate=serverCertificate.crt --tls-key=serverPrivateKey.key` |
    | AWS Lambda Function  | `N/A`                                                                                                                                                                                                      |
 
 3. Configure the Prometheus Connector to listen for Prometheus requests on `http://localhost:3080`.
 
    | Runtime              | Command                                                                                                                                                        |
    | -------------------- |----------------------------------------------------------------------------------------------------------------------------------------------------------------|
-   | Precompiled Binaries | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=PrometheusDatabase  --default-table=PrometheusMetricsTable --web.listen-address=:3080` |
+   | Precompiled Binaries | `./bootstrap --default-database=PrometheusDatabase  --default-table=PrometheusMetricsTable --web.listen-address=:3080` |
    | AWS Lambda Function  | `N/A`                                                                                                                                                          |
 
 4. Configure the Prometheus Connector to listen for Prometheus requests on `http://localhost:3080` and serve collected metrics to `http://localhost:3080/timestream-metrics`.
 
    | Runtime              | Command                                                                                                                                                                                                 |
    | -------------------- |---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-   | Precompiled Binaries | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=PrometheusDatabase  --default-table=PrometheusMetricsTable --web.listen-address=:3080 --web.telemetry-path=/timestream-metrics` |
+   | Precompiled Binaries | `./bootstrap --default-database=PrometheusDatabase  --default-table=PrometheusMetricsTable --web.listen-address=:3080 --web.telemetry-path=/timestream-metrics` |
    | AWS Lambda Function  | `N/A`                                                                                                                                                                                                   |
 
 ### Retry Configuration Options
@@ -582,7 +578,7 @@ Configure the Prometheus Connector to retry up to 10 times upon recoverable erro
 
 | Runtime              | Command                                                                                                                                                                                           |
 | -------------------- |---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| Precompiled Binaries | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=PrometheusDatabase  --default-table=PrometheusMetricsTable --max-retries=10`                                              |
+| Precompiled Binaries | `./bootstrap --default-database=PrometheusDatabase  --default-table=PrometheusMetricsTable --max-retries=10`                                              |
 | AWS Lambda Function  | `aws lambda update-function-configuration --function-name PrometheusConnector --environment "Variables={default_database=prometheusDatabase,default_table=prometheusMetricsTable,max_retries=10}"` |
 
 ### Logger Configuration Options
@@ -610,21 +606,21 @@ To quickly spot and resolve issues that may be caused by ignored Prometheus time
 
    | Runtime              | Command                                                                                                                                                                                                            |
    | -------------------- |--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-   | Precompiled Binaries | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=PrometheusDatabase  --default-table=PrometheusMetricsTable --enable-logging=false`                                                         |
+   | Precompiled Binaries | `./bootstrap --default-database=PrometheusDatabase  --default-table=PrometheusMetricsTable --enable-logging=false`                                                         |
    | AWS Lambda Function  | `aws lambda update-function-configuration --function-name PrometheusPrometheus Connector --environment "Variables={default_database=prometheusDatabase,default_table=prometheusMetricsTable,enable_logging=false}"` |
 
 2. Toggle the Prometheus Connector to halt on: <br />- label names exceeding the maximum length supported by Amazon Timestream;<br />- Prometheus time series with non-finite values.
 
    | Runtime              | Command                                                                                                                                                                                                                                                           |
    | -------------------- |-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-   | Precompiled Binaries | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=PrometheusDatabase  --default-table=PrometheusMetricsTable --fail-on-long-label=true --fail-on-invalid-sample=true` |
+   | Precompiled Binaries | `./bootstrap --default-database=PrometheusDatabase  --default-table=PrometheusMetricsTable --fail-on-long-label=true --fail-on-invalid-sample=true` |
    | AWS Lambda Function  | `aws lambda update-function-configuration --function-name PrometheusConnector --environment "Variables={default_database=prometheusDatabase,default_table=prometheusMetricsTable,fail_on_long_label=true, fail_on_invalid_sample_value=true}"`                     |
 
 3. Configure the Prometheus Connector to output the logs at debug level and in JSON format.
 
     | Runtime              | Command                                                                                                                                                                                                             |
     | -------------------- |---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-    | Precompiled Binaries | `./timestream-prometheus-connector-linux-amd64-1.0.0 --default-database=PrometheusDatabase  --default-table=PrometheusMetricsTable --log.level=debug --log.format=json`                                             |
+    | Precompiled Binaries | `./bootstrap --default-database=PrometheusDatabase  --default-table=PrometheusMetricsTable --log.level=debug --log.format=json`                                             |
     | AWS Lambda Function  | `aws lambda update-function-configuration --function-name PrometheusConnector --environment "Variables={default_database=prometheusDatabase,default_table=prometheusMetricsTable,log_level=debug, log_format=json}"` |
 
 ## Relabel Long Labels

diff --git a/main.go b/main.go
@@ -117,14 +117,14 @@ func main() {
 		awsWriteConfigs.MaxRetries = aws.Int(writeClientMaxRetries)
 		timestreamClient.NewWriteClient(logger, awsWriteConfigs, cfg.failOnLongMetricLabelName, cfg.failOnInvalidSample)
 
-		timestream.LogInfo(logger, "Successfully created Timestream clients to handle read and write requests from Prometheus.")
-
+		timestream.LogInfo(logger, fmt.Sprintf("Timestream connection is initialized (Database: %s, Table: %s, Region: %s)", cfg.defaultDatabase, cfg.defaultTable, cfg.clientConfig.region))
 		// Register TimestreamClient to Prometheus for it to scrape metrics
 		prometheus.MustRegister(timestreamClient)
 
 		writers = append(writers, timestreamClient.WriteClient())
 		readers = append(readers, timestreamClient.QueryClient())
 
+		timestream.LogInfo(logger, "The Prometheus Connector is now ready to begin serving ingestion and query requests.")
 		if err := serve(logger, cfg.listenAddr, writers, readers, cfg.certificate, cfg.key); err != nil {
 			timestream.LogError(logger, "Error occurred while listening for requests.", err)
 			os.Exit(1)
@@ -184,8 +184,7 @@ func handleWriteRequest(reqBuf []byte, timestreamClient *timestream.Client, awsC
 
 	createWriteClient(timestreamClient, logger, awsConfigs, cfg.failOnLongMetricLabelName, cfg.failOnInvalidSample)
 
-	timestream.LogInfo(logger, "Successfully created a Timestream write client to handle write requests from Prometheus.")
-
+	timestream.LogInfo(logger, fmt.Sprintf("Timestream write connection is initialized (Database: %s, Table: %s, Region: %s)", cfg.defaultDatabase, cfg.defaultTable, cfg.clientConfig.region))
 	if err := getWriteClient(timestreamClient).Write(&writeRequest, credentials); err != nil {
 		errorCode := http.StatusBadRequest
 
@@ -214,7 +213,7 @@ func handleReadRequest(reqBuf []byte, timestreamClient *timestream.Client, awsCo
 
 	createQueryClient(timestreamClient, logger, awsConfigs, cfg.maxRetries)
 
-	timestream.LogInfo(logger, "Successfully created a Timestream query client to handle write requests from Prometheus.")
+	timestream.LogInfo(logger, fmt.Sprintf("Timestream query connection is initialized (Database: %s, Table: %s, Region: %s)", cfg.defaultDatabase, cfg.defaultTable, cfg.clientConfig.region))
 
 	response, err := getQueryClient(timestreamClient).Read(&readRequest, credentials)
 	if err != nil {
@@ -386,6 +385,16 @@ func parseFlags() *connectionConfig {
 		os.Exit(1)
 	}
 
+	if cfg.defaultDatabase == "" {
+		kingpin.Errorf("The default database value must be set through the flag --default-database")
+		os.Exit(1)
+	}
+
+	if cfg.defaultTable == "" {
+		kingpin.Errorf("The default table value must be set through the flag --default-table")
+		os.Exit(1)
+	}
+
 	return cfg
 }