Merge branch 'feat-2.0' into debian-package-updates

Cargo.toml

@@ -15,7 +15,7 @@ anyhow = "1"
 async-stream = "0.3.4"
 async-trait = "0.1.77"
 casper-types = { git = "https://github.com/casper-network/casper-node.git", branch = "feat-2.0" }
-casper-binary-port = { git = "https://github.com/casper-network/casper-node.git", branch = "feat-2.0"  }
+casper-binary-port = { git = "https://github.com/casper-network/casper-node.git", branch = "feat-2.0" }
 casper-event-sidecar = { path = "./event_sidecar", version = "1.0.0" }
 casper-event-types = { path = "./types", version = "1.0.0" }
 casper-rpc-sidecar = { path = "./rpc_sidecar", version = "1.0.0" }

event_sidecar/src/database/writer_generator.rs

@@ -312,8 +312,7 @@ network_name: String,
         event_id: u32,
         event_source_address: String,
         api_version: String,
-network_name: String,
-
+        network_name: String,
     ) -> Result<u64, DatabaseWriteError> {
         #[cfg(feature = "additional-metrics")]
         let start = Instant::now();

event_sidecar/src/utils.rs

@@ -263,14 +263,14 @@ pub mod tests {
         config: TestingConfig,
     ) -> tokio::task::JoinHandle<Result<ExitCode, Error>> {
         tokio::spawn(async move { unpack_test_config_and_run(config, true).await })
-        // starting event sidecar
+        // starting the sidecar
     }
 
     pub async fn start_sidecar(
         config: TestingConfig,
     ) -> tokio::task::JoinHandle<Result<ExitCode, Error>> {
         tokio::spawn(async move { unpack_test_config_and_run(config, false).await })
-        // starting event sidecar
+        // starting the sidecar
     }
 
     pub fn build_test_config() -> (TestingConfig, TempDir, u16, u16, u16) {

json_rpc/README.md

@@ -1,4 +1,4 @@
-# `casper-json-rpc`
+# The `casper-json-rpc` Library
 
 [![LOGO](https://raw.githubusercontent.com/casper-network/casper-node/master/images/casper-association-logo-primary.svg)](https://casper.network/)
 
@@ -7,16 +7,15 @@
 [![Documentation](https://docs.rs/casper-node/badge.svg)](https://docs.rs/casper-json-rpc)
 [![License](https://img.shields.io/badge/license-Apache-blue)](https://github.com/casper-network/casper-node/blob/master/LICENSE)
 
-A library suitable for use as the framework for a JSON-RPC server.
+The `casper-json-rpc` library described here can be used as the framework for a JSON-RPC server.
 
 # Usage
 
-Normally usage will involve two steps:
-  * construct a set of request handlers using a
-    [`RequestHandlersBuilder`](https://docs.rs/casper-json-rpc/latest/casper_json_rpc/struct.RequestHandlersBuilder.html)
-  * call [`casper_json_rpc::route`](https://docs.rs/casper-json-rpc/latest/casper_json_rpc/fn.route.html) to construct a
-    boxed warp filter ready to be passed to [`warp::service`](https://docs.rs/warp/latest/warp/fn.service.html) for
-    example
+Typical usage of this library involves two steps:
+
+* Construct a set of request handlers using a
+[`RequestHandlersBuilder`](https://docs.rs/casper-json-rpc/latest/casper_json_rpc/struct.RequestHandlersBuilder.html).
+* Call [`casper_json_rpc::route`](https://docs.rs/casper-json-rpc/latest/casper_json_rpc/fn.route.html) to construct a boxed warp filter ready to be passed to [`warp::service`](https://docs.rs/warp/latest/warp/fn.service.html).
 
 # Example
 
@@ -61,29 +60,28 @@ async fn main() {
 }
 ```
 
-If this receives a request such as
+The following is a sample request:
 
-```
+```sh
 curl -X POST -H 'Content-Type: application/json' -d '{"jsonrpc":"2.0","id":"id","method":"get"}' http://127.0.0.1:3030/rpc
 ```
 
-then the server will respond with
+Here is a sample response:
 
-```json
+```sh
 {"jsonrpc":"2.0","id":"id","result":"got it"}
 ```
 
 # Errors
 
 To return a JSON-RPC response indicating an error, use
 [`Error::new`](https://docs.rs/casper-json-rpc/latest/casper_json_rpc/struct.Error.html#method.new).  Most error
-conditions which require returning a reserved error are already handled in the provided warp filters.  The only
+conditions that require returning a reserved error are already handled in the provided warp filters.  The only
 exception is
-[`ReservedErrorCode::InvalidParams`](https://docs.rs/casper-json-rpc/latest/casper_json_rpc/enum.ReservedErrorCode.html#variant.InvalidParams)
-which should be returned by any RPC handler which deems the provided `params: Option<Params>` to be invalid for any
+[`ReservedErrorCode::InvalidParams`](https://docs.rs/casper-json-rpc/latest/casper_json_rpc/enum.ReservedErrorCode.html#variant.InvalidParams), which should be returned by any RPC handler that deems the provided `params: Option<Params>` to be invalid for any
 reason.
 
-Generally a set of custom error codes should be provided.  These should all implement
+Generally, a set of custom error codes should be provided.  These should all implement
 [`ErrorCodeT`](https://docs.rs/casper-json-rpc/latest/casper_json_rpc/trait.ErrorCodeT.html).
 
 ## Example custom error code

resources/ETC_README.md

@@ -1,237 +1,74 @@
-# Casper Event Sidecar README for Node Operators
+# Casper Sidecar README for Node Operators
 
-## Summary of Purpose
+This page contains specific instructions for node operators. Before proceeding, familiarize yourself with the main [README](../README.md) file, which covers the following:
+ - [Summary of purpose](../README.md#summary-of-purpose)
+ - [System components and architecture](../README.md#system-components-and-architecture)
+ - [Configuration options](../README.md#configuring-the-sidecar)
+ - [Running and testing the Sidecar](../README.md#running-and-testing-the-sidecar)
+ - [Troubleshooting tips](../README.md#troubleshooting-tips)
 
-The Casper Event Sidecar is an application that runs in tandem with the node process. This reduces the load on the node process by allowing subscribers to monitor the event stream through the Sidecar, while the node focuses entirely on the blockchain. Users needing access to the JSON-RPC will still need to query the node directly.
 
-While the primary use case for the Sidecar application is running alongside the node on the same machine, it can be run remotely if necessary.
-
-### System Components & Architecture
-
-Casper Nodes offer a Node Event Stream API returning Server-Sent Events (SSEs) that hold JSON-encoded data. The SSE Sidecar uses this API to achieve the following goals:
-
-* Build a sidecar middleware service that reads the Event Stream of all connected nodes, acting as a passthrough and replicating the SSE interface of the connected nodes and their filters (i.e., `/main`, `/deploys`, and `/sigs` with support for the use of the `?start_from=` query to allow clients to get previously sent events from the Sidecar's buffer).
-
-* Provide a new RESTful endpoint that is discoverable to node operators.
-
-The SSE Sidecar uses one ring buffer for outbound events, providing some robustness against unintended subscriber disconnects. If a disconnected subscriber re-subscribes before the buffer moves past their last received event, there will be no gap in the event history if they use the `start_from` URL query.
-
-
-## Configuration
+## Configuring the Sidecar
 
 The file `/etc/casper-sidecar/config.toml` holds a default configuration. This should work if installed on a Casper node.
 
 If you install the Sidecar on an external server, you must update the `ip-address` values under `node_connections` appropriately.
 
-### Node Connections
-
-The Sidecar can connect to Casper nodes with versions greater or equal to `2.0.0`.
-
-The `node_connections` option configures the node (or multiple nodes) to which the Sidecar will connect and the parameters under which it will operate with that node.
-
-```
-[[sse_server.connections]]
-ip_address = "127.0.0.1"
-sse_port = 9999
-rest_port = 8888
-max_attempts = 10
-delay_between_retries_in_seconds = 5
-allow_partial_connection = false
-enable_logging = true
-connection_timeout_in_seconds = 3
-no_message_timeout_in_seconds = 60
-sleep_between_keep_alive_checks_in_seconds = 30
-```
-
-* `ip_address` - The IP address of the node to monitor.
-* `sse_port` - The node's event stream (SSE) port. This [example configuration](../resources/example_configs/EXAMPLE_NODE_CONFIG.toml) uses port `9999`.
-* `rest_port` - The node's REST endpoint for status and metrics. This [example configuration](../resources/example_configs/EXAMPLE_NODE_CONFIG.toml) uses port `8888`.
-* `max_attempts` - The maximum number of attempts the Sidecar will make to connect to the node. If set to `0`, the Sidecar will not attempt to connect.
-* `delay_between_retries_in_seconds` - The delay between attempts to connect to the node.
-* `allow_partial_connection` - Determining whether the sidecar will allow a partial connection to this node.
-* `enable_logging` - This enables logging of events from the node in question.
-* `connection_timeout_in_seconds` - Number of seconds before the connection request times out. Parameter is optional, defaults to 5
-* `no_message_timeout_in_seconds` - Number of seconds after which the connection will be restarted if no bytes were received. Parameter is optional, defaults to 120
-* `sleep_between_keep_alive_checks_in_seconds` - Optional parameter specifying the time intervals (in seconds) for checking if the connection is still alive. Defaults to 60
-
-Connecting to multiple nodes requires multiple `[[sse_server.connections]]` sections:
-
-```
-[[sse_server.connections]]
-ip_address = "127.0.0.1"
-sse_port = 9999
-rest_port = 8888
-max_attempts = 10
-delay_between_retries_in_seconds = 5
-allow_partial_connection = false
-enable_logging = true
-
-[[sse_server.connections]]
-ip_address = "18.154.79.193"
-sse_port = 1234
-rest_port = 3456
-max_attempts = 10
-delay_between_retries_in_seconds = 5
-allow_partial_connection = false
-enable_logging = true
-```
-
-### Storage
+For more information, including how to setup the SSE, RPC, REST, and Admin servers, read the [configuration options](../README.md#configuring-the-sidecar) in the main README.
 
-This directory stores the SSE cache and an SQLite database if the Sidecar is configured to use SQLite.
 
-```
-[storage]
-storage_path = "/var/lib/casper-sidecar"
-```
-
-### Database Connectivity
-
-<!--TODO for the Database Connectivity section, we could point to the Github README -->
-
-The Sidecar can connect to different types of databases. The current options are `SQLite` or `PostgreSQL`. The following sections show how to configure the database connection for one of these DBs. Note that the Sidecar can only connect to one DB at a time.
-
-#### SQLite Database
-
-This section includes configurations for the SQLite database.
-
-```
-[storage.sqlite_config]
-file_name = "sqlite_database.db3"
-max_connections_in_pool = 100
-# https://www.sqlite.org/compile.html#default_wal_autocheckpoint
-wal_autocheckpointing_interval = 1000
-```
-
-* `file_name` - The database file path.
-* `max_connections_in_pool` - The maximum number of connections to the database. (Should generally be left as is.)
-* `wal_autocheckpointing_interval` - This controls how often the system commits pages to the database. The value determines the maximum number of pages before forcing a commit. More information can be found [here](https://www.sqlite.org/compile.html#default_wal_autocheckpoint).
-
-#### PostgreSQL Database
-
-The properties listed below are elements of the PostgreSQL database connection that can be configured for the Sidecar.
-
-* `database_name` - Name of the database.
-* `host` - URL to PostgreSQL instance.
-* `database_username` - Username.
-* `database_password` - Database password.
-* `max_connections_in_pool` - The maximum number of connections to the database.
-* `port` - The port for the database connection.
-
-
-To run the Sidecar with PostgreSQL, you can set the following database environment variables to control how the Sidecar connects to the database. This is the suggested method to set the connection information for the PostgreSQL database.
-
-```
-SIDECAR_POSTGRES_USERNAME="your username"
-```
+## Installing the Sidecar on a Node
 
-```
-SIDECAR_POSTGRES_PASSWORD="your password"
-```
+The following command will install the Debian package for the Casper Sidecar service on various flavors of Linux. 
 
-```
-SIDECAR_POSTGRES_DATABASE_NAME="your database name"
-```
+<!-- TODO Once the package is published, update the command below with the new link to the *.deb package. The link below assumes a package available locally. -->
 
-```
-SIDECAR_POSTGRES_HOST="your host"
+```bash
+sudo apt install ./casper-sidecar_0.1.0-0_amd64.deb
 ```
 
-```
-SIDECAR_POSTGRES_MAX_CONNECTIONS="max connections"
-```
+Check the service status:
 
+```bash
+systemctl status casper-sidecar
 ```
-SIDECAR_POSTGRES_PORT="port"
-```
-
-However, DB connectivity can also be configured using the Sidecar configuration file.
-
-If the DB environment variables and the Sidecar's configuration file have the same variable set, the DB environment variables will take precedence.
 
-It is possible to completely omit the PostgreSQL configuration from the Sidecar's configuration file. In this case, the Sidecar will attempt to connect to the PostgreSQL using the database environment variables or use some default values for non-critical variables.
+Check the logs and make sure the service is running as expected.
 
+```bash
+journalctl --no-pager -u casper-sidecar
 ```
-[storage.postgresql_config]
-database_name = "event_sidecar"
-host = "localhost"
-database_password = "p@$$w0rd"
-database_username = "postgres"
-max_connections_in_pool = 30
-```
-
-### REST & Event Stream Criteria
-
-This information determines outbound connection criteria for the Sidecar's `rest_server`.
 
-<!--TODO for the REST & Event Stream Criteria section, we could point to the Github README -->
+If you see any errors, you may need to [update the configuration](#configuring-the-service) and restart the service with the commands below.
 
-```
-[rest_api_server]
-port = 18888
-max_concurrent_requests = 50
-max_requests_per_second = 50
-request_timeout_in_seconds = 10
-```
+## Running the Sidecar on a Node
 
-* `port` - The port for accessing the Sidecar's `rest_server`. `18888` is the default, but operators are free to choose their own port as needed.
-* `max_concurrent_requests` - The maximum total number of simultaneous requests that can be made to the REST server.
-* `max_requests_per_second` - The maximum total number of requests that can be made per second.
-* `request_timeout_in_seconds` - The total time before a request times out.
+The `casper-sidecar` service starts after installation, using the systemd service file.
 
-```
-[event_stream_server]
-port = 19999
-max_concurrent_subscribers = 100
-event_stream_buffer_length = 5000
-```
+### Stop
 
-The `event_stream_server` section specifies a port for the Sidecar's event stream.
+`sudo systemctl stop casper-sidecar.service`
 
-Additionally, there are the following two options:
+### Start
 
-* `max_concurrent_subscribers` - The maximum number of subscribers that can monitor the Sidecar's event stream.
-* `event_stream_buffer_length` - The number of events that the stream will hold in its buffer for reference when a subscriber reconnects.
+`sudo systemctl start casper-sidecar.service`
 
-### Admin Server
 
-<!--TODO for the Admin Server section, we could point to the Github README -->
+## Sidecar Storage
 
-This optional section configures the Sidecar's administrative REST server. If this section is not specified, the Sidecar will not start an admin server.
+This directory stores the SSE cache and a database if the Sidecar was configured to use one.
 
-```
-[admin_api_server]
-port = 18887
-max_concurrent_requests = 1
-max_requests_per_second = 1
+```toml
+[storage]
+storage_path = "/var/lib/casper-sidecar"
 ```
 
-* `port` - The port for accessing the Sidecar's admin REST server.
-* `max_concurrent_requests` - The maximum total number of simultaneous requests that can be sent to the admin server.
-* `max_requests_per_second` - The maximum total number of requests that can be sent per second to the admin server.
-
-Access the admin server at `http://localhost:18887/metrics/`.
+The DB setup is described [here](../README#database-connectivity-setup).
 
 ## Swagger Documentation
 
-Once the Sidecar is running, access the Swagger documentation at `http://localhost:18888/swagger-ui/`.
+If the Sidecar is running locally, access the Swagger documentation at `http://localhost:18888/swagger-ui/`.
 
 ## OpenAPI Specification
 
-An OpenAPI schema is available at `http://localhost:18888/api-doc.json/`.
-
-## Running the Event Sidecar
-
-The `casper-sidecar` service starts after installation, using the systemd service file.
-
-### Stop
-
-`sudo systemctl stop casper-sidecar.service`
-
-### Start
-
-`sudo systemctl start casper-sidecar.service`
-
-### Logs
-
-`journalctl --no-pager -u casper-sidecar`
+An OpenAPI schema is available at `http://localhost:18888/api-doc.json/`.