Create RPC middleware using RPC services #5290

mcmire · 2025-02-06T00:30:36Z

Explanation

When creating the middleware stack for a network, NetworkController makes use of functions from two packages for sending requests:

createInfuraMiddleware from @metamask/eth-json-rpc-infura for Infura networks
createFetchMiddleware from @metamask/eth-json-rpc-middleware for custom RPC endpoints

Currently, each of these middleware implements similar heuristics for determining when the network is down and retrying requests appropriately. We want to improve upon this logic so that we incorporate the circuit breaker pattern, exponential backoff, and jitter for retries, and, in the case of Infura, we can fail over to an alternate RPC endpoint when it is down. To make this happen, we have implemented an RpcService class which extracts all of this logic into one place. As a result, we can vastly simplify the Infura and fetch middleware creation functions so that they only need to take an RPC service and the RPC service will handle the rest.

To accommodate these changes, this commit:

Upgrades the @metamask/eth-json-rpc-infura and @metamask/eth-json-rpc-middleware packages to allow an RPC service to be passed.
Adds two new required options to NetworkController, fetch and btoa. These are also requirements of RpcService and allows consumers to pass in whatever versions of these functions is specific to their platform.
Updates createAutoManagedNetworkClient to take fetch and btoa and pass them along to createNetworkClient.
Updates createNetworkClient to create an RPC service and pass it along to the two middleware creation functions. Consequently, these functions now handle requests in exactly the same way.

References

Closes #4991.

Changelog

(Updated in PR.)

Checklist

I've updated the test suite for new or updated code as appropriate
I've updated documentation (JSDoc, Markdown, etc.) for new or updated code as appropriate
I've highlighted breaking changes using the "BREAKING" category above as appropriate
I've prepared draft pull requests for clients and consumer packages to resolve any breaking changes

socket-security · 2025-02-12T20:28:22Z

Removed dependencies detected. Learn more about Socket for GitHub ↗︎

🚮 Removed packages: npm/@metamask/[email protected]

View full report↗︎

When creating the middleware stack for a network, NetworkController makes use of functions from two packages for sending requests: - `createInfuraMiddleware` from `@metamask/eth-json-rpc-infura` for Infura networks - `createFetchMiddleware` from `@metamask/eth-json-rpc-middleware` for custom RPC endpoints Currently, each of these middleware implements similar heuristics for determining when the network is down and retrying requests appropriately. We want to improve upon this logic so that we incorporate the circuit breaker pattern, exponential backoff, and jitter for retries, and we have implemented an RpcService class which extracts all of this logic into one place. We can vastly simplify the Infura and fetch middleware creation functions so that they only need to take an RPC service and the RPC service will handle the rest. To accommodate these changes, this commit: - Upgrades the `@metamask/eth-json-rpc-infura` and `@metamask/eth-json-rpc-middleware` packages to allow an RPC service to be passed. - Adds two new required options to NetworkController, `fetch` and `btoa`. These are also requirements of RpcService and allows consumers to pass in whatever versions of these functions is specific to their platform. - Updates `createAutoManagedNetworkClient` to take `fetch` and `btoa` and pass them along to `createNetworkClient`. - Updates `createNetworkClient` to create an RPC service and pass it along to the two middleware creation functions. Consequently, these functions now handle requests in exactly the same way.

mikesposito

Great work!

mikesposito · 2025-02-13T15:28:39Z

packages/network-controller/src/create-auto-managed-network-client.ts

+ * @param args.fetch - A function that can be used to make an HTTP request,
+ * compatible with the Fetch API.
+ * @param args.btoa - A function that can be used to convert a binary string
+ * into base-64.


nit:

Suggested change

* into base-64.

* into a base64-encoded ASCII string.

Ah, makes sense. I plan on making a sort of cleanup PR later so I will make this change then.

cryptodev-2s

LGTM!

mcmire · 2025-02-12T21:38:29Z

packages/network-controller/tests/NetworkController.test.ts

There are lots of changes to tests here but this is because of the changes to the signature of the NetworkController constructor and also createNetworkClient.

mcmire · 2025-02-12T21:42:15Z

packages/network-controller/tests/provider-api-tests/block-param.ts

@@ -385,167 +380,45 @@ export function testsForRpcMethodSupportingBlockParam(
      });
    });

-    // There is a difference in how we are testing the Infura middleware vs. the


This commit consolidates the request handling logic between the Infura and fetch middleware, so they now have the same behavior, and as a result, we can simplify the tests that exercise the middleware.

mcmire · 2025-02-12T21:42:56Z

packages/network-controller/tests/provider-api-tests/helpers.ts

@@ -195,6 +194,10 @@ function mockRpcCall({
    // TODO: Replace `any` with type
    // eslint-disable-next-line @typescript-eslint/no-explicit-any
    return nockRequest.reply(httpStatus, (_, requestBody: any) => {
+      if (typeof completeResponse === 'string') {


In tests we now need the ability to completely override the response body to test for what happens when the response is not JSON-parseable.

mcmire · 2025-02-12T21:43:39Z

packages/network-controller/tests/provider-api-tests/no-block-param.ts

@@ -265,114 +260,32 @@ export function testsForRpcMethodAssumingNoBlockParam(
    });
  });

-  // There is a difference in how we are testing the Infura middleware vs. the


Similar as block-param.ts: We can simplify these tests because both the Infura and fetch middleware now handle requests in the same way.

mcmire · 2025-02-13T22:26:51Z

packages/network-controller/src/create-auto-managed-network-client.ts

+ * @param args.fetch - A function that can be used to make an HTTP request,
+ * compatible with the Fetch API.
+ * @param args.btoa - A function that can be used to convert a binary string
+ * into base-64.


Ah, makes sense. I plan on making a sort of cleanup PR later so I will make this change then.

OGPoyraz

Confirmation changes LGTM

mcmire force-pushed the integrate-rpc-failover-into-nc-2 branch from 5a4f6f1 to 62bea3b Compare February 6, 2025 18:41

mcmire changed the base branch from main to include-node-fetch-errors-as-retriable February 6, 2025 21:19

mcmire force-pushed the integrate-rpc-failover-into-nc-2 branch 2 times, most recently from 5b7e301 to 40fa2ca Compare February 6, 2025 21:25

Base automatically changed from include-node-fetch-errors-as-retriable to main February 12, 2025 16:43

mcmire force-pushed the integrate-rpc-failover-into-nc-2 branch from 40fa2ca to eee47d9 Compare February 12, 2025 20:27

mcmire changed the title ~~WIP - Integrate RPC failover into NetworkController~~ Create RPC middleware using RPC services Feb 12, 2025

mcmire force-pushed the integrate-rpc-failover-into-nc-2 branch from eee47d9 to da61e84 Compare February 12, 2025 21:37

mcmire force-pushed the integrate-rpc-failover-into-nc-2 branch from da61e84 to e690c89 Compare February 12, 2025 21:41

mcmire marked this pull request as ready for review February 12, 2025 21:46

mcmire requested review from a team as code owners February 12, 2025 21:46

mcmire added the team-wallet-framework label Feb 12, 2025

mikesposito approved these changes Feb 13, 2025

View reviewed changes

cryptodev-2s approved these changes Feb 13, 2025

View reviewed changes

Merge branch 'main' into integrate-rpc-failover-into-nc-2

31ccb0f

mcmire enabled auto-merge (squash) February 13, 2025 22:27

mcmire commented Feb 13, 2025

View reviewed changes

OGPoyraz approved these changes Feb 14, 2025

View reviewed changes

Prithpal-Sooriya approved these changes Feb 14, 2025

View reviewed changes

mathieuartu approved these changes Feb 14, 2025

View reviewed changes

Merge branch 'main' into integrate-rpc-failover-into-nc-2

3bb9acf

mcmire merged commit f8c8a90 into main Feb 14, 2025
133 checks passed

mcmire deleted the integrate-rpc-failover-into-nc-2 branch February 14, 2025 16:05

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create RPC middleware using RPC services #5290

Create RPC middleware using RPC services #5290

mcmire commented Feb 6, 2025 •

edited

Loading

socket-security bot commented Feb 12, 2025 •

edited

Loading

mikesposito left a comment

mikesposito Feb 13, 2025 •

edited

Loading

mcmire Feb 13, 2025

cryptodev-2s left a comment

mcmire Feb 12, 2025

mcmire Feb 12, 2025

mcmire Feb 12, 2025

mcmire Feb 12, 2025

mcmire Feb 13, 2025

OGPoyraz left a comment

Create RPC middleware using RPC services #5290

Create RPC middleware using RPC services #5290

Conversation

mcmire commented Feb 6, 2025 • edited Loading

Explanation

References

Changelog

Checklist

socket-security bot commented Feb 12, 2025 • edited Loading

mikesposito left a comment

Choose a reason for hiding this comment

mikesposito Feb 13, 2025 • edited Loading

Choose a reason for hiding this comment

mcmire Feb 13, 2025

Choose a reason for hiding this comment

cryptodev-2s left a comment

Choose a reason for hiding this comment

mcmire Feb 12, 2025

Choose a reason for hiding this comment

mcmire Feb 12, 2025

Choose a reason for hiding this comment

mcmire Feb 12, 2025

Choose a reason for hiding this comment

mcmire Feb 12, 2025

Choose a reason for hiding this comment

mcmire Feb 13, 2025

Choose a reason for hiding this comment

OGPoyraz left a comment

Choose a reason for hiding this comment

mcmire commented Feb 6, 2025 •

edited

Loading

socket-security bot commented Feb 12, 2025 •

edited

Loading

mikesposito Feb 13, 2025 •

edited

Loading