Support EKS Pod Identity (alternative to IRSA)

[thomascjohnson]

Is your feature request related to a problem or challenge? Please describe what you are trying to do.

I have a rust binary that makes use of object_store and I'd like to use it in AWS EKS with Pod Identity to access objects in S3. Maybe I'm missing something, but it seems like object_store's AWS S3 setup can't make use of the credentials provided by Pod Identity. Looking here, I see the reference to these environment variables:

AWS_ACCESS_KEY_ID -> access_key_id
AWS_SECRET_ACCESS_KEY -> secret_access_key
AWS_DEFAULT_REGION -> region
AWS_ENDPOINT -> endpoint
AWS_SESSION_TOKEN -> token
AWS_CONTAINER_CREDENTIALS_RELATIVE_URI -> https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html
AWS_ALLOW_HTTP -> set to “true” to permit HTTP connections without TLS

Pod Identity makes use of different variables per this documentation:

AWS_CONTAINER_CREDENTIALS_FULL_URI=http://169.254.170.23/v1/credentials
AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE=/var/run/secrets/pods.eks.amazonaws.com/serviceaccount/eks-pod-identity-token

Am I missing something or is it currently not possible to make use of these credentials with object_store?

Describe the solution you'd like
I'd like to be able to use the Pod Identity credentials with object_store without having to do any additional workarounds.

Describe alternatives you've considered
I would like to avoid putting an access key and secret in a Kubernetes secret and mounting that to the containers. Other than that, I'm looking for a way to minimally adapt my code with something that can use the container credentials URI and auth token file to authenticate with object_store's S3 store.

Additional context
I'm happy to contribute to the project, although I would need a bit of guidance to understand how to adapt the library for this issue.

[tustvold]

For those following along, it looks like AWS added a new form of EKS authentication - https://aws.amazon.com/blogs/containers/amazon-eks-pod-identity-a-new-way-for-applications-on-eks-to-obtain-iam-credentials/.

object_store already supports the older style IRSA based EKS authentication.

Adding support for EKS pod identity is likely relatively straightforward and we'd welcome PRs to add support for it