[Epic] Have all manifests in example YAML be able to go through Uptest

[muvaf]

What problem are you facing?

We have examples manifests in each provider but not all of them can work in Uptest.

How could Official Providers help solve your problem?

There are roughly the following classes of resources:

• Examples of inherited resources from Jet providers.
  • These were assumed to be working since we had merged them with manual testing but that's not the case for all of them.
• Examples we merged after a report of manual testing.
  • We manually tested and reported the results in the PR body before Uptest was ready to be used for all scenarios.
• Examples that require manual intervention, like file upload, but not marked as such.

Ideally, we should make all manifests be able to go through Uptest without errors so that we can run the pipeline whenever we need to, like TF provide upgrade, and not get false negatives.

In order to reach that, we should do a pass over them to fix bugs and mark the ones that require manual intervention with uptest.upbound.io/manual-intervention annotation to tell users what they need to do and to instruct Uptest to skip that specific resource during testing. Later, we can improve Uptest to take those actions and remove the annotations over time. But first, we need to get to a place where all examples can be run with Uptest without error.

This is an epic issue, related current issues are:

• Test inherited resources:
  • https://github.com/upbound/official-providers/issues/410
  • https://github.com/upbound/official-providers/issues/549
  • https://github.com/upbound/official-providers/issues/550
  • https://github.com/upbound/official-providers/issues/501
  • https://github.com/upbound/official-providers/issues/502
• Test the rest:
  • https://github.com/upbound/official-providers/issues/545
  • aws: Run Uptest for all examples folder except Jet-inherited -Part II- crossplane-contrib/provider-upjet-aws#15
  • https://github.com/upbound/official-providers/issues/546
  • https://github.com/upbound/official-providers/issues/547
• Uptest improvements:
  • https://github.com/upbound/official-providers/issues/399
  • https://github.com/upbound/official-providers/issues/548
  • https://github.com/upbound/official-providers/issues/605
  • https://github.com/upbound/official-providers/issues/655

More issues will be needed for the rest of the resources until /test-examples="provider-{aws,gcp,azure}/examples" works.