RPC sidecar changes (#231)

* Moving rpc sidecar to event sidecar workspace. Both og those servers will work on one binary * Updating schemars version because the old one is having bugs (if there is a name collision for components schema only one will be saved) * Copying casper_types to this project. The reason is that casper_types in release <= 4.0.1 depends on old schemars * Copying casper_types for 2.0 release to this project. The reason is that rpc sidecar has to use the new types definitions, but for now they are not released (and it's not clear if they will be released prior to node 2.0 release). * Changing RpcError implementation to fix tests. Some alignments of codestyle to make clippy happy. * Moving casper-types dependencies to workspace level * Sync changes from node branch * Update the schema file * Delete protocol.md * Move a DbId fix * Change error message * Changes to versioning * Sync changes to types * Switch to having a single binary * Moving config files, fixing compilation issues * bump 'juliet' to '0.2.1' * Sync casper-types changes * Changing RPC sidecar config so that the rpc_server.node_client.exponential_backoff will take a new parameter called max_attempts. I tcan be either "infinite" or a positive, non-zero number. * Storing ApiVersion in event_log table. Removing is_big_integer_id config from DDLConfiguration because it's no longer needed (new version of sea_query handles the situation of defining big_integer and autoincrement) * Revert "Storing ApiVersion in event_log table. Removing is_big_integer_id con…" * Update for node review changes (#15) * Update for node changes * Fix lints * Cleanup * Cover all values in tag roundtrip tests * Moving admin server out from the sse sidecar. They are spinned up separately from sse events server. Also the database initialization happens separetely. Is sse events server is defined a storage definition is required. If rest api server is defined a storage definition is required. * Fix GlobalStateRequest::random * Changes explicit BoxFuture casting to calling 'boxed()' method --------- Co-authored-by: Jakub Zajkowski <[email protected]> Co-authored-by: Rafał Chabowski <[email protected]> Co-authored-by: Rafał Chabowski <[email protected]> Co-authored-by: zajko <[email protected]>
casper-network · Feb 23, 2024 · 7511324 · 7511324
1 parent d1d5150
commit 7511324
Show file tree

Hide file tree

Showing 436 changed files with 113,025 additions and 1,629 deletions.
diff --git a/Cargo.lock b/Cargo.lock
diff --git a/Cargo.toml b/Cargo.toml
@@ -1,12 +1,30 @@
 [workspace]
 resolver = "1"
 members = [
-    "sidecar",
+    "casper_types",
+    "casper_types_ver_2_0",
+    "event_sidecar",
+    "json_rpc",
     "listener",
-    "types",
+    "rpc_sidecar",
+    "sidecar",
+    "types"
 ]
 
 [workspace.dependencies]
-once_cell = "1.18.0"
+anyhow = "1"
 async-stream = "0.3.4"
+casper-types = { path = "./casper_types", version = "4.0.1" }
+casper-types-ver-2_0 = { version = "3.0.0", path = "./casper_types_ver_2_0" }
+casper-event-sidecar = { path = "./event_sidecar", version = "1.0.0" }
+casper-rpc-sidecar = { path = "./rpc_sidecar", version = "1.0.0" }
+datasize = "0.2.11"
+futures = "0"
 futures-util = "0.3.28"
+once_cell = "1.18.0"
+thiserror = "1"
+tokio = "1.23.1"
+toml = "0.5.8"
+tracing = { version = "0", default-features = false }
+tracing-subscriber = "0"
+serde = { version = "1", default-features = false }
diff --git a/README.md b/README.md
@@ -32,20 +32,20 @@ The SSE Sidecar service must be configured using a `.toml` file specified at run
 
 This repository contains several sample configuration files that can be used as examples and adjusted according to your scenario:
 
-- [EXAMPLE_NCTL_CONFIG.toml](./EXAMPLE_NCTL_CONFIG.toml) - Configuration for connecting to nodes on a local NCTL network. This configuration is used in the unit and integration tests found in this repository
-- [EXAMPLE_NCTL_POSTGRES_CONFIG.toml](./EXAMPLE_NCTL_POSTGRES_CONFIG.toml) - Configuration for using the PostgreSQL database and nodes on a local NCTL network
-- [EXAMPLE_NODE_CONFIG.toml](./EXAMPLE_NODE_CONFIG.toml) - Configuration for connecting to live nodes on a Casper network and setting up an admin server
+- [EXAMPLE_NCTL_CONFIG.toml](./resources/example_configs/EXAMPLE_NCTL_CONFIG.toml) - Configuration for connecting to nodes on a local NCTL network. This configuration is used in the unit and integration tests found in this repository
+- [EXAMPLE_NCTL_POSTGRES_CONFIG.toml](./resources/example_configs/EXAMPLE_NCTL_POSTGRES_CONFIG.toml) - Configuration for using the PostgreSQL database and nodes on a local NCTL network
+- [EXAMPLE_NODE_CONFIG.toml](./resources/example_configs/EXAMPLE_NODE_CONFIG.toml) - Configuration for connecting to live nodes on a Casper network and setting up an admin server    
 
 Once you create the configuration file and are ready to run the Sidecar service, you must provide the configuration as an argument using the `-- --path-to-config` option as described [here](#running-the-sidecar).
 
 ### Node Connections
 
 The Sidecar can connect to Casper nodes with versions greater or equal to `1.5.2`.
 
-The `node_connections` option configures the node (or multiple nodes) to which the Sidecar will connect and the parameters under which it will operate with that node. Connecting to multiple nodes requires multiple `[[connections]]` sections.
+The `node_connections` option configures the node (or multiple nodes) to which the Sidecar will connect and the parameters under which it will operate with that node. Connecting to multiple nodes requires multiple `[[sse_server.connections]]` sections.
 
 ```
-[[connections]]
+[[sse_server.connections]]
 ip_address = "127.0.0.1"
 sse_port = 18101
 rest_port = 14101
@@ -57,7 +57,7 @@ connection_timeout_in_seconds = 3
 no_message_timeout_in_seconds = 60
 sleep_between_keep_alive_checks_in_seconds = 30
 
-[[connections]]
+[[sse_server.connections]]
 ip_address = "127.0.0.1"
 sse_port = 18102
 rest_port = 14102
@@ -69,7 +69,7 @@ connection_timeout_in_seconds = 3
 no_message_timeout_in_seconds = 60
 sleep_between_keep_alive_checks_in_seconds = 30
 
-[[connections]]
+[[sse_server.connections]]
 ip_address = "127.0.0.1"
 sse_port = 18103
 rest_port = 14103
@@ -83,8 +83,8 @@ sleep_between_keep_alive_checks_in_seconds = 30
 ```
 
 * `ip_address` - The IP address of the node to monitor.
-* `sse_port` - The node's event stream (SSE) port. This [example configuration](EXAMPLE_NODE_CONFIG.toml) uses port `9999`.
-* `rest_port` - The node's REST endpoint for status and metrics. This [example configuration](EXAMPLE_NODE_CONFIG.toml) uses port `8888`.
+* `sse_port` - The node's event stream (SSE) port. This [example configuration](./resources/example_configs/EXAMPLE_NODE_CONFIG.toml) uses port `9999`.
+* `rest_port` - The node's REST endpoint for status and metrics. This [example configuration](./resources/example_configs/EXAMPLE_NODE_CONFIG.toml) uses port `8888`.
 * `max_attempts` - The maximum number of attempts the Sidecar will make to connect to the node. If set to `0`, the Sidecar will not attempt to connect.
 * `delay_between_retries_in_seconds` - The delay between attempts to connect to the node.
 * `allow_partial_connection` - Determining whether the Sidecar will allow a partial connection to this node.
@@ -180,7 +180,7 @@ max_connections_in_pool = 30
 This information determines outbound connection criteria for the Sidecar's `rest_server`.
 
 ```
-[rest_server]
+[rest_api_server]
 port = 18888
 max_concurrent_requests = 50
 max_requests_per_second = 50
@@ -193,7 +193,7 @@ request_timeout_in_seconds = 10
 * `request_timeout_in_seconds` - The total time before a request times out.
 
 ```
-[event_stream_server]
+[sse_server.event_stream_server]
 port = 19999
 max_concurrent_subscribers = 100
 event_stream_buffer_length = 5000
@@ -211,7 +211,7 @@ Additionally, there are the following two options:
 This optional section configures the Sidecar's administrative server. If this section is not specified, the Sidecar will not start an admin server.
 
 ```
-[admin_server]
+[admin_api_server]
 port = 18887
 max_concurrent_requests = 1
 max_requests_per_second = 1
@@ -245,22 +245,22 @@ You can also run the performance tests using the following command:
 cargo test -- --include-ignored
 ```
 
-The [EXAMPLE_NCTL_CONFIG.toml](./EXAMPLE_NCTL_CONFIG.toml) file contains the configurations used for these tests.
+The [EXAMPLE_NCTL_CONFIG.toml](./resources/example_configs/EXAMPLE_NCTL_CONFIG.toml) file contains the configurations used for these tests.
 
 ## Running the Sidecar
 
 After creating the configuration file, run the Sidecar using Cargo and point to the configuration file using the `--path-to-config` option, as shown below. The command needs to run with `root` privileges.
 
 ```shell
-sudo cargo run -- --path-to-config EXAMPLE_NODE_CONFIG.toml
+sudo cargo run -- --path-to-config ./resources/example_configs/EXAMPLE_NODE_CONFIG.toml
 ```
 
 The Sidecar application leverages tracing, which can be controlled by setting the `RUST_LOG` environment variable.
 
 The following command will run the sidecar application with the `INFO` log level.
 
 ```
-RUST_LOG=info cargo run -p casper-event-sidecar -- --path-to-config EXAMPLE_NCTL_CONFIG.toml
+RUST_LOG=info cargo run -p casper-event-sidecar -- --path-to-config ./resources/example_configs/EXAMPLE_NCTL_CONFIG.toml
 ```
 
 The log levels, listed in order of increasing verbosity, are:

diff --git a/USAGE.md b/USAGE.md
@@ -35,7 +35,7 @@ curl -s http://<HOST:PORT>/events/<TYPE>
 - `PORT` - The port number where the Sidecar emits events
 - `TYPE` - The type of event emitted
 
-Given this [example configuration](EXAMPLE_NODE_CONFIG.toml), here are the commands for each endpoint:
+Given this [example configuration](./resources/example_configs/EXAMPLE_NODE_CONFIG.toml), here are the commands for each endpoint:
 
 - **Deploy events:** 
 

diff --git a/casper_types/CHANGELOG.md b/casper_types/CHANGELOG.md
@@ -0,0 +1,200 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.  The format is based on [Keep a Changelog].
+
+[comment]: <> (Added:      new features)
+[comment]: <> (Changed:    changes in existing functionality)
+[comment]: <> (Deprecated: soon-to-be removed features)
+[comment]: <> (Removed:    now removed features)
+[comment]: <> (Fixed:      any bug fixes)
+[comment]: <> (Security:   in case of vulnerabilities)
+
+
+
+## 4.0.1
+
+### Added
+* Add a new `SyncHandling` enum, which allows a node to opt out of historical sync.
+
+### Changed
+* Update `k256` to version 0.13.1.
+
+### Removed
+* Remove `ExecutionResult::successful_transfers`.
+
+### Security
+* Update `ed25519-dalek` to version 2.0.0 as mitigation for [RUSTSEC-2022-0093](https://rustsec.org/advisories/RUSTSEC-2022-0093)
+
+
+
+## 3.0.0
+
+### Added
+* Add new `bytesrepr::Error::NotRepresentable` error variant that represents values that are not representable by the serialization format.
+* Add new `Key::Unbond` key variant under which the new unbonding information (to support redelegation) is written.
+* Add new `Key::ChainspecRegistry` key variant under which the `ChainspecRegistry` is written.
+* Add new `Key::ChecksumRegistry` key variant under which a registry of checksums for a given block is written.  There are two checksums in the registry, one for the execution results and the other for the approvals of all deploys in the block.
+* Add new `StoredValue::Unbonding` variant to support redelegating.
+* Add a new type `WithdrawPurses` which is meant to represent `UnbondingPurses` as they exist in current live networks.
+
+### Changed
+* Extend `UnbondingPurse` to take a new field `new_validator` which represents the validator to whom tokens will be re-delegated.
+* Increase `DICTIONARY_ITEM_KEY_MAX_LENGTH` to 128.
+* Change prefix of formatted string representation of `ContractPackageHash` from "contract-package-wasm" to "contract-package-". Parsing from the old format is still supported.
+* Apply `#[non_exhaustive]` to error enums.
+* Change Debug output of `DeployHash` to hex-encoded string rather than a list of integers.
+
+### Fixed
+* Fix some integer casts, where failure is now detected and reported via new error variant `NotRepresentable`.
+
+
+
+## 2.0.0
+
+### Fixed
+* Republish v1.6.0 as v2.0.0 due to missed breaking change in API (addition of new variant to `Key`).
+
+
+
+## 1.6.0 [YANKED]
+
+### Added
+* Extend asymmetric key functionality, available via feature `std` (moved from `casper-nodes` crate).
+* Provide `Timestamp` and `TimeDiff` types for time operations, with extended functionality available via feature `std` (moved from `casper-nodes` crate).
+* Provide test-only functionality, in particular a seedable RNG `TestRng` which outputs its seed on test failure. Available via a new feature `testing`.
+* Add new `Key::EraSummary` key variant under which the era summary info is written on each switch block execution.
+
+### Deprecated
+* Deprecate `gens` feature: its functionality is included in the new `testing` feature.
+
+
+
+## 1.5.0
+
+### Added
+* Provide types and functionality to support improved access control inside execution engine.
+* Provide `CLTyped` impl for `ContractPackage` to allow it to be passed into contracts.
+
+### Fixed
+* Limit parsing of CLTyped objects to a maximum of 50 types deep.
+
+
+
+## 1.4.6 - 2021-12-29
+
+### Changed
+* Disable checksummed-hex encoding, but leave checksummed-hex decoding in place.
+
+
+
+## 1.4.5 - 2021-12-06
+
+### Added
+* Add function to `auction::MintProvider` trait to support minting into an existing purse.
+
+### Changed
+* Change checksummed hex implementation to use 32 byte rather than 64 byte blake2b digests.
+
+
+
+## [1.4.4] - 2021-11-18
+
+### Fixed
+* Revert the accidental change to the `std` feature causing a broken build when this feature is enabled.
+
+
+
+## [1.4.3] - 2021-11-17 [YANKED]
+
+
+
+## [1.4.2] - 2021-11-13 [YANKED]
+
+### Added
+* Add checksummed hex encoding following a scheme similar to [EIP-55](https://eips.ethereum.org/EIPS/eip-55).
+
+
+
+## [1.4.1] - 2021-10-23
+
+No changes.
+
+
+
+## [1.4.0] - 2021-10-21 [YANKED]
+
+### Added
+* Add `json-schema` feature, disabled by default, to enable many types to be used to produce JSON-schema data.
+* Add implicit `datasize` feature, disabled by default, to enable many types to derive the `DataSize` trait.
+* Add `StoredValue` types to this crate.
+
+### Changed
+* Support building and testing using stable Rust.
+* Allow longer hex string to be presented in `json` files. Current maximum is increased from 100 to 150 characters.
+* Improve documentation and `Debug` impls for `ApiError`.
+
+### Deprecated
+* Feature `std` is deprecated as it is now a no-op, since there is no benefit to linking the std lib via this crate.
+
+
+
+## [1.3.0] - 2021-07-19
+
+### Changed
+* Restrict summarization when JSON pretty-printing to contiguous long hex strings.
+* Update pinned version of Rust to `nightly-2021-06-17`.
+
+### Removed
+* Remove ability to clone `SecretKey`s.
+
+
+
+## [1.2.0] - 2021-05-27
+
+### Changed
+* Change to Apache 2.0 license.
+* Return a `Result` from the constructor of `SecretKey` rather than potentially panicking.
+* Improve `Key` error reporting and tests.
+
+### Fixed
+* Fix `Key` deserialization.
+
+
+
+## [1.1.1] - 2021-04-19
+
+No changes.
+
+
+
+## [1.1.0] - 2021-04-13 [YANKED]
+
+No changes.
+
+
+
+## [1.0.1] - 2021-04-08
+
+No changes.
+
+
+
+## [1.0.0] - 2021-03-30
+
+### Added
+* Initial release of types for use by software compatible with Casper mainnet.
+
+
+
+[Keep a Changelog]: https://keepachangelog.com/en/1.0.0
+[unreleased]: https://github.com/casper-network/casper-node/compare/24fc4027a...dev
+[1.4.3]: https://github.com/casper-network/casper-node/compare/2be27b3f5...24fc4027a
+[1.4.2]: https://github.com/casper-network/casper-node/compare/v1.4.1...2be27b3f5
+[1.4.1]: https://github.com/casper-network/casper-node/compare/v1.4.0...v1.4.1
+[1.4.0]: https://github.com/casper-network/casper-node/compare/v1.3.0...v1.4.0
+[1.3.0]: https://github.com/casper-network/casper-node/compare/v1.2.0...v1.3.0
+[1.2.0]: https://github.com/casper-network/casper-node/compare/v1.1.1...v1.2.0
+[1.1.1]: https://github.com/casper-network/casper-node/compare/v1.0.1...v1.1.1
+[1.1.0]: https://github.com/casper-network/casper-node/compare/v1.0.1...v1.1.1
+[1.0.1]: https://github.com/casper-network/casper-node/compare/v1.0.0...v1.0.1
+[1.0.0]: https://github.com/casper-network/casper-node/releases/tag/v1.0.0