Reasoning behind "internal"-izing endpoint structures

[yawn]

In v1 enumerating partitions for e.g. service / region-for-service availability was supported and documented e.g. here https://aws.amazon.com/blogs/developer/using-the-aws-sdk-for-gos-regions-and-endpoints-metadata/

Any particular reason why this functionality has been abandoned?

[yawn]

Is this the right place to ask questions / start a discussion? Or should I migrate this to an issue?

[jasdel]

Yeah this is a OK place to ask questions, and discuss around the behavior of the SDK. behavior of the SDK. Discussions can always later be created as Issues if bugs, or changes are needed.

[jasdel]

Thanks for reaching out @yawn. The V2 SDK intentional made endpoint metadata an internal component of the SDK. The endpoint metadata was made internal component of the API client modules for a few reasons.

Modularization: When the SDK's API clients were split off into their own modules, we wanted to limit the metadata that they shared. The primary reason for this was to ensure that users of the SDK didn't run into an issue if they update the module containing endpoints without updating the API client module as well. This lead us to split the endpoint metadata into each API client's module. Splitting these into separate modules also helped a little bit with application start since the application's no longer need to initialize a very large data structure for the endpoints for all APIs. This was an issue in v1 SDK. aws/aws-sdk-go#3717

Staleness of metadata: For the use case of determining what AWS Regions are available, Amazon EC2's DescribeRegions operation would be much better to use to dynamically determine the regions that are available. The SDK's metadata is a fixed point in time and will be of date when new regions are added until the SDK is updated in the application.

The SDK will always attempt to resolve an endpoint for the AWS region string provided, even if that region isn't enumerated in the SDK's endpoint metadata. In many cases, APIs did not model all regions they were available in making the endpoint metadata inaccurate for determining which regions an API is available in.

Stable metadata: The metadata content and structure generated into the SDKs is intended to hint to the SDK what endpoint to use when communicating with an API. This information can change. Such as the ServiceID in the metadata changing in the v1 SDK causing the SDK to stop generating a set of constants for the known APIs, aws/aws-sdk-go#2338.

[yawn]

Thanks a lot for the clarification @jasdel. The primary use case for this metadata previously was the question "in what regions is service X available ∩ regions which are currently enable (using EC2 DescribeRegions)". This information is no longer available through the SDK and has to be retrieved e.g. via https://api.regional-table.region-services.aws.a2z.com/index.json (without the ability to match directly to clients).

From my limited POV the staleness of metadata is not really a problem. If the SDK is too old I'm missing out of e.g. new API calls or features as well.

IMHO clients should expose partition and subsequently region availablity directly.

[yawn]

As an alternative I can use https://github.com/yawn/geographer but this will require additional (CI) efforts not to become outdated and has the additional problem of not being directly integrated into clients. Is the metadata drift such a problem that this feature has no chance to become part of the v2 SDK?

[github-actions[bot]]

Hello! Reopening this discussion to make it searchable.