Skip to content

Commit

Permalink
Merge branch 'main' into eric/ics-compat
Browse files Browse the repository at this point in the history
  • Loading branch information
Alex Johnson authored Apr 11, 2024
2 parents 05eb941 + f06be79 commit 284a15d
Show file tree
Hide file tree
Showing 20 changed files with 51 additions and 55 deletions.
3 changes: 2 additions & 1 deletion abci/strategies/currencypair/mocks/mock_oracle_keeper.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

24 changes: 12 additions & 12 deletions oracle/config/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,17 +8,17 @@
>
> <div align="center">
>
> | Type | Oracle/Market Config | Oracle Metrics | App Metrics |
> |----------|:--------:|---------:|--------:|
> | Validator | **Required** | **Recommended** | **Recommended** |
> | Non-Validator | **Optional** | **Optional** | **Optional** |
> | Type | Oracle/Market Config | Oracle Metrics | App Metrics |
> |---------------|:--------------------:|----------------:|----------------:|
> | Validator | **Required** | **Recommended** | **Recommended** |
> | Non-Validator | **Optional** | **Optional** | **Optional** |
> </div>
All oracle configurations are broken down into three files:

1. **Oracle side-car configuration (`oracle.json`):** This contains the data provider's that are utilized, how often they should be polled, and a variety of other configurations for API and web socket providers.
2. **Market side-car configuration (`market.json`):** This contains the desired markets that the side-car will fetch prices for. NOTE: It is recommended that this file is **NOT** modified nor created by validators. This file is typically provided by the chain that the side-car supports.
3. **Oracle configuration in the application (`app.toml`):** A few additional lines of code that must be added to the application's `app.toml` file to configure the oracle side car into the application.
3. **Oracle configuration in the application (`app.toml`):** A few additional lines of code that must be added to the application's `app.toml` file to configure the oracle sidecar into the application.

*The focus of this readme is the oracle side-car configuration and the application configuration. The market side-car configuration is typically provided by the chain that the oracle supports.*

Expand All @@ -29,7 +29,7 @@ The `app.toml` file is the configuration file that is consumed by the applicatio
```toml
# Other configurations

...
# ...

###############################################################################
### Oracle ###
Expand Down Expand Up @@ -58,7 +58,7 @@ metrics_enabled = "{{ .Oracle.MetricsEnabled }}"
# exposed to.
prometheus_server_address = "{{ .Oracle.PrometheusServerAddress }}"

...
# ...

# More configurations
```
Expand All @@ -68,7 +68,7 @@ In your `app.toml`, you should see / write something that looks like this.
> Note: This is only required if you are running a validator node. If you are running a non-validator node, you can skip this section.
```toml
...
# ...


###############################################################################
Expand Down Expand Up @@ -98,12 +98,12 @@ metrics_enabled = "true"
# exposed to.
prometheus_server_address = "0.0.0.0:8001"

...
# ...
```

# Oracle Side-Car Configuration

The `oracle.json` file is the configuration file that is consumed by the oracle side-car. Note that in most cases, this should **NOT** be custom made by validators - unless specified otherwise. A predefined oracle side car configuration should be provided by the chain that the oracle supports. This file contains:
The `oracle.json` file is the configuration file that is consumed by the oracle sidecar. Note that in most cases, this should **NOT** be custom-made by validators - unless specified otherwise. A predefined oracle sidecar configuration should be provided by the chain that the oracle supports. This file contains:

* The desired data providers to be utilized i.e. Coinbase, Binance, etc.
* Metrics instrumentation.
Expand Down Expand Up @@ -186,7 +186,7 @@ This field is utilized to set the amount of time the provider should wait for a

#### Interval

This field is utilized to set the interval at which the provider should update the prices. Note that provider's may rate limit based on this interval so it is recommended to tune this value as necessary.
This field is utilized to set the interval at which the provider should update the prices. Note that provider's may rate limit based on this interval, so it is recommended to tune this value as necessary.

#### MaxQueries

Expand Down Expand Up @@ -746,6 +746,6 @@ Sample configuration:

# Conclusion

This readme has provided an overview of how to configure the oracle side-car and application. It has also provided a brief overview of the oracle side-car configuration and the application configuration. To see an example of a properly configured oracle side car, please visit the [local config](./../../config/local) files - `oracle.json` and `market.json`.
This readme has provided an overview of how to configure the oracle side-car and application. It has also provided a brief overview of the oracle side-car configuration and the application configuration. To see an example of a properly configured oracle sidecar, please visit the [local config](./../../config/local) files - `oracle.json` and `market.json`.

In general, it is best to consult the chain's documentation and channels to determine the correct configurations for the oracle side-car. If you have any questions, please feel free to reach out to the Skip team on the [Skip Discord](https://discord.com/invite/hFeHVAE26P).
2 changes: 1 addition & 1 deletion oracle/config/api.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ type APIConfig struct {
}

// Endpoint holds all data necessary for an API provider to connect to a given endpoint
// i.e URL, headers, authentication, etc.
// i.e. URL, headers, authentication, etc.
type Endpoint struct {
// URL is the URL that is used to fetch data from the API.
URL string `json:"url"`
Expand Down
2 changes: 1 addition & 1 deletion oracle/config/app.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ oracle_address = "{{ .Oracle.OracleAddress }}"
client_timeout = "{{ .Oracle.ClientTimeout }}"
# MetricsEnabled determines whether oracle metrics are enabled. Specifically
# this enables intsrumentation of the oracle client and the interaction between
# this enables instrumentation of the oracle client and the interaction between
# the oracle and the app.
metrics_enabled = "{{ .Oracle.MetricsEnabled }}"
Expand Down
2 changes: 1 addition & 1 deletion oracle/config/metrics.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ import (
)

// MetricsConfig is the metrics configurations for the oracle. This configuration object specifically
// exposes metrics pertaining to the oracle side car. To enable app side metrics, please see the app
// exposes metrics pertaining to the oracle sidecar. To enable app side metrics, please see the app
// configuration.
type MetricsConfig struct {
// PrometheusServerAddress is the address of the prometheus server that the oracle will expose
Expand Down
2 changes: 1 addition & 1 deletion oracle/metrics/metrics.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ type OracleMetricsImpl struct {
providerTick *prometheus.CounterVec
}

// NewMetricsFromConfig returns a oracle Metrics implementation based on the provided
// NewMetricsFromConfig returns an oracle Metrics implementation based on the provided
// config.
func NewMetricsFromConfig(config config.MetricsConfig) Metrics {
if config.Enabled {
Expand Down
4 changes: 1 addition & 3 deletions oracle/metrics_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,6 @@ type OracleMetricsTestSuite struct {

const (
oracleTicker = 1 * time.Second
provider1 = "provider1"
provider2 = "provider2"
)

func TestOracleMetricsTestSuite(t *testing.T) {
Expand Down Expand Up @@ -97,7 +95,7 @@ func (s *OracleMetricsTestSuite) TestTickMetric() {

// wait for a tick on the oracle
go func() {
s.o.Start(context.Background())
s.Require().NoError(s.o.Start(context.Background()))
}()

// wait for a tick
Expand Down
6 changes: 3 additions & 3 deletions oracle/orchestrator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,9 @@ The Orchestrator is the central component of the oracle. It is responsible for:

## Configuration

At a high level the orchestrator is configured with a `oracle.json` file that contains all of the providers that need to be instantiated. To read more about the configuration of `oracle.json`, please refer to the [oracle configuration documentation](./../config/README.md).
At a high level the orchestrator is configured with a `oracle.json` file that contains all providers that need to be instantiated. To read more about the configuration of `oracle.json`, please refer to the [oracle configuration documentation](./../config/README.md).

Each provider is instantiate using the `PriceAPIQueryHandlerFactory`, `PriceWebSocketQueryHandlerFactory`, and `MarketMapFactory` factory functions. Think of these as the constructors for the providers.
Each provider is instantiated using the `PriceAPIQueryHandlerFactory`, `PriceWebSocketQueryHandlerFactory`, and `MarketMapFactory` factory functions. Think of these as the constructors for the providers.

* `PriceAPIQueryHandlerFactory` - This is used to create the API query handler for the provider - which is then passed into a base provider.
* `PriceWebSocketQueryHandlerFactory` - This is used to create the WebSocket query handler for the provider - which is then passed into a base provider.
Expand All @@ -22,5 +22,5 @@ The orchestrator can be initialized with an option of `WithMarketMap` which allo

The orchestrator will then start each provider in a separate goroutine. Additionally, if the orchestrator has a market map provider, it will start a goroutine that will periodically fetch the markets from the market map provider and update the providers accordingly.

All of the providers are running concurrently and will do so until the main context is canceled (what is passed into `Start`). If the orchestrator is canceled, it will cancel all of the providers and wait for them to finish before returning.
All providers are running concurrently and will do so until the main context is canceled (what is passed into `Start`). If the orchestrator is canceled, it will cancel all providers and wait for them to finish before returning.

2 changes: 1 addition & 1 deletion oracle/orchestrator/init.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import (
mmclienttypes "github.com/skip-mev/slinky/service/clients/marketmap/types"
)

// Init initializes the all of the providers that are configured via the oracle config.
// Init initializes the all providers that are configured via the oracle config.
func (o *ProviderOrchestrator) Init() error {
o.mut.Lock()
defer o.mut.Unlock()
Expand Down
6 changes: 3 additions & 3 deletions oracle/orchestrator/init_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -159,11 +159,11 @@ func TestInit(t *testing.T) {
t.Run("errors when a provider is not supported by the web socket query handler factory", func(t *testing.T) {
cfg := oracleCfg

okx := okx.DefaultWebSocketConfig
okx.Name = "unsupported"
okxCfg := okx.DefaultWebSocketConfig
okxCfg.Name = "unsupported"
cfg.Providers = append(cfg.Providers, config.ProviderConfig{
Name: "unsupported",
WebSocket: okx,
WebSocket: okxCfg,
Type: oracletypes.ConfigType,
})

Expand Down
8 changes: 4 additions & 4 deletions oracle/orchestrator/lifecycle.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ import (
"go.uber.org/zap"
)

// generalProvider is a interface for a provider that implements the base provider.
// generalProvider is an interface for a provider that implements the base provider.
type generalProvider interface {
// Start starts the provider.
Start(ctx context.Context) error
// Stop stops the provider.
// Name is the provider's name.
Name() string
}

Expand All @@ -27,7 +27,7 @@ func (o *ProviderOrchestrator) Start(ctx context.Context) error {
// Set the main context for the provider orchestrator.
ctx, _ = o.setMainCtx(ctx)

// Start all of the price providers.
// Start all price providers.
for _, state := range o.providers {
o.wg.Add(1)
go func() {
Expand Down Expand Up @@ -55,7 +55,7 @@ func (o *ProviderOrchestrator) Start(ctx context.Context) error {
}

// Stop stops the provider orchestrator. This is a synchronous operation that will
// wait for all of the providers to exit.
// wait for all providers to exit.
func (o *ProviderOrchestrator) Stop() {
o.logger.Info("stopping provider orchestrator")
if _, cancel := o.getMainCtx(); cancel != nil {
Expand Down
6 changes: 3 additions & 3 deletions oracle/orchestrator/lifecycle_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ func TestStart(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go func() {
o.Start(ctx)
require.NoError(t, o.Start(ctx))
}()

time.Sleep(5 * time.Second)
Expand All @@ -61,7 +61,7 @@ func TestStart(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go func() {
o.Start(ctx)
require.NoError(t, o.Start(ctx))
}()

time.Sleep(5 * time.Second)
Expand All @@ -85,7 +85,7 @@ func TestStart(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go func() {
o.Start(ctx)
require.NoError(t, o.Start(ctx))
}()

time.Sleep(5 * time.Second)
Expand Down
2 changes: 1 addition & 1 deletion oracle/orchestrator/options.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ func WithPriceAPIQueryHandlerFactory(factory types.PriceAPIQueryHandlerFactory)
}
}

// WithWebSocketQueryHandlerFactory sets the websocket query handler factory for the provider orchestrator.
// WithPriceWebSocketQueryHandlerFactory sets the websocket query handler factory for the provider orchestrator.
// Specifically, this is what is utilized to construct price providers that are websocket based.
func WithPriceWebSocketQueryHandlerFactory(factory types.PriceWebSocketQueryHandlerFactory) Option {
return func(m *ProviderOrchestrator) {
Expand Down
8 changes: 4 additions & 4 deletions oracle/orchestrator/orchestrator.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ import (
)

// ProviderOrchestrator is a stateful orchestrator that is responsible for maintaining
// all of the providers that the oracle is using. This includes initializing the providers,
// all providers that the oracle is using. This includes initializing the providers,
// creating the provider specific market map, and enabling/disabling the providers based
// on the oracle configuration and market map.
type ProviderOrchestrator struct {
Expand All @@ -35,7 +35,7 @@ type ProviderOrchestrator struct {

// -------------------Stateful Fields-------------------//
//
// providers is a map of all of the providers that the oracle is using.
// providers is a map of all providers that the oracle is using.
providers map[string]ProviderState
// mmProvider is the market map provider. Specifically this provider is responsible
// for making requests for the latest market map data.
Expand Down Expand Up @@ -107,15 +107,15 @@ func NewProviderOrchestrator(
return orchestrator, nil
}

// GetProviderState returns all of the providers and their state.
// GetProviderState returns all providers and their state.
func (o *ProviderOrchestrator) GetProviderState() map[string]ProviderState {
o.mut.Lock()
defer o.mut.Unlock()

return o.providers
}

// GetPriceProviders returns all of the price providers.
// GetPriceProviders returns all price providers.
func (o *ProviderOrchestrator) GetPriceProviders() []*types.PriceProvider {
o.mut.Lock()
defer o.mut.Unlock()
Expand Down
2 changes: 1 addition & 1 deletion oracle/orchestrator/update.go
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ func (o *ProviderOrchestrator) UpdateWithMarketMap(marketMap mmtypes.MarketMap)
return err
}

// Iterate over all of the existing providers and update their market maps.
// Iterate over all existing providers and update their market maps.
for name, state := range o.providers {
providerTickers, err := types.ProviderTickersFromMarketMap(name, marketMap)
if err != nil {
Expand Down
8 changes: 4 additions & 4 deletions oracle/orchestrator/update_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -303,7 +303,7 @@ func TestUpdateProviderState(t *testing.T) {

// Start the provider.
go func() {
o.Start(ctx)
require.NoError(t, o.Start(ctx))
}()

time.Sleep(500 * time.Millisecond)
Expand Down Expand Up @@ -388,7 +388,7 @@ func TestUpdateProviderState(t *testing.T) {

// Start the provider.
go func() {
o.Start(ctx)
require.NoError(t, o.Start(ctx))
}()

time.Sleep(1000 * time.Millisecond)
Expand Down Expand Up @@ -475,7 +475,7 @@ func TestUpdateProviderState(t *testing.T) {

// Start the provider.
go func() {
o.Start(ctx)
require.NoError(t, o.Start(ctx))
}()

time.Sleep(3 * time.Millisecond)
Expand Down Expand Up @@ -566,7 +566,7 @@ func TestUpdateProviderState(t *testing.T) {

// Start the provider.
go func() {
o.Start(ctx)
require.NoError(t, o.Start(ctx))
}()

time.Sleep(1000 * time.Millisecond)
Expand Down
5 changes: 3 additions & 2 deletions oracle/providers_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,8 @@ func (s *OracleTestSuite) TestProviders() {

for _, provider := range providers {
go func() {
provider.Start(ctx)
// context deadline exceeded
s.Require().Error(provider.Start(ctx))
}()
}

Expand All @@ -159,7 +160,7 @@ func (s *OracleTestSuite) TestProviders() {
s.Require().NoError(err)

go func() {
testOracle.Start(ctx)
s.Require().NoError(testOracle.Start(ctx))
}()

// Wait for the oracle to start and update.
Expand Down
4 changes: 2 additions & 2 deletions oracle/types/market.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,10 @@ func ProviderTickersFromMarketMap(
}

var (
// Track all of the tickers that the provider will be providing data for.
// Track all tickers that the provider will be providing data for.
providerTickers = make([]ProviderTicker, 0)
// Maintain a set of off-chain tickers that have been seen to avoid duplicates.
// Notably, the side-car provider enforces a uniquness constraint for off-chain tickers.
// Notably, the side-car provider enforces a uniqueness constraint for off-chain tickers.
seenOffChainTickers = make(map[string]struct{})
)

Expand Down
6 changes: 1 addition & 5 deletions oracle/types/oracle.go
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ var (
// NewPriceResult is a function alias for the new price result.
NewPriceResult = providertypes.NewResult[*big.Float]

// NewPricesResponse is a function alias for the new price response.
// NewPriceResponse is a function alias for the new price response.
NewPriceResponse = providertypes.NewGetResponse[ProviderTicker, *big.Float]

// NewPriceResponseWithErr is a function alias for the new price response with errors.
Expand All @@ -92,10 +92,6 @@ var (
// NewPriceProvider is a function alias for the new price provider.
NewPriceProvider = base.NewProvider[ProviderTicker, *big.Float]

// NewPriceAPIQueryHandler is a function alias for the new API query handler meant to be
// used by the price providers.
NewPriceAPIQueryHandler = apihandlers.NewAPIQueryHandler[ProviderTicker, *big.Float]

// NewPriceAPIQueryHandlerWithFetcher is a function alias for the new API query handler with fetcher.
NewPriceAPIQueryHandlerWithFetcher = apihandlers.NewAPIQueryHandlerWithFetcher[ProviderTicker, *big.Float]

Expand Down
Loading

0 comments on commit 284a15d

Please sign in to comment.