[Bug]: PodIdentityAssociation errors with ResourceInUseException: Association already exists #1437
Comments
|
i tested v1.11.0 without any issues: and i can see the associations with the correct ID in aws console 
|
haarchri
added
is:triaged
|
After updating EKS Provider to v1.11.0, I've yet to reproduce this issue as well. |
|
After several days of testing, ran into this issue again while using v1.11.0. |
|
Still experiencing this issue with v1.14.0 release |
|
I'm experiencing this, too. I'm running crossplane 1.16.2, eks provider 1.11 and aws provider family 1.16.0. Deleting the podidentityassociation in AWS fixes the issue. However, I need to delete the resource in Kubernetes first, then quickly trigger the delete in AWS. As extra information: I'm running all pods with two replicas as only customization. It's also not with every pia, but when creating ten or so, it happens. |
|
This provider repo does not have enough maintainers to address every issue. Since there has been no activity in the last 90 days it is now marked as |
|
/fresh |
|
I'm experiencing this issue as well in v1.19.0. |
|
This is easy to replicate:
Resuming, we cant import existing resources. Seems there's an issue when the upbound tries to match the state with the object. |
|
@pintonunes that's not expected to work unless you set the generated ID as external-name, something like this: |
|
Yes.. You're right @chlunde . Sorry for the confusion. So, after lots of troubleshooting we were able to reproduce the issue. When the MRs got stuck the status looks like this:
We notice that the provider-upjet-aws/config/externalname.go Lines 3227 to 3234 in 245b952 and the Every time we face this is issue we also have write conflicts: This seems like a classic issue when the id is generated by the cloud provider. In this case, to get the resource we need the right association_id and the cluster_name. So very briefly, on the first reconcile the provider goes thru the Connect/Observe (using a stub assoc id cause its needed by the AWS API) and checks that the resource doesnt exists. Then goes to the Create and so on, is able to create the resource, sets the external name but is unable to store the updated status (due to conflict). So the association id that will be there is the stub one. On the next reconciles, the provider will the use the stub assoc id (on status) to get the resource and will not find it. So, will try to create again. Since the resource already exists it will error out and here's where the endless loop starts. Basically because the association id is wrong.. I prepared a PR with the fix. We already tested and everything seems to be working. |
Is there an existing issue for this?
Affected Resource(s)
PodIdentityAssociation.eks.aws.upbound.io/v1beta1
Resource MRs required to reproduce the bug
No response
Steps to Reproduce
This issue appear to randomly affect newly created PodIdentityAssociations.
What happened?
New
PodIdentityAssociationis applied and confirm creation in AWS EKS console. ButPodIdentityAssociationSynced and Ready states immediately turn toFalseand the following conditions are returned.One observation with these failed PodIdentityAssociations is, the
status.atProvider.associationId, iea-stubassocid123456, is set to a value that does not correspond to any Identity Associations with the AWS console or within any provider-aws-eks Pod logs. Also, when I have more than one failedPodIdentityAssociation, thestatus.atProvider.associationIdall match,a-stubassocid123456.Searching the codebase, this value only appears in one location:
provider-upjet-aws/config/externalname.go
Line 3171 in 779097a
If I delete the Identity Association within the AWS console, the resource is eventually reconciled, recreated within the console, but PodIdentityAssociation enters the same failed Synced and Ready states. Only after deleting the Identity Association from AWS console and
PodIdentityAssociationfrom Kubernetes, Composite Resource creates a newPodIdentityAssociationmanaged resource, the Identity Association is created in AWS console andPodIdentityAssociationenters a Synced and Ready state ofTrue.Relevant Error Output Snippet
Crossplane Version
1.16.0
Provider Version
1.10.0
Kubernetes Version
No response
Kubernetes Distribution
EKS
Additional Info
No response
The text was updated successfully, but these errors were encountered: