Call provider API from controller workflows

[stefania-popescu]

Overview

Currently, the argo workflows that manage CRUD operations for resources call the provider CLI during the provider step. Now it needs to be changed so that the workflow calls the provider-service API. Each CLI call should have a corresponding API call that needs to be configured.

There's an integration test that stubs out the (old) provider interface; when the provider-service API is called, the new provider interface will need to be stubbed out since the shape is slightly different.

Tasks

• Create stubbed provider service for testing
• Change provider CLI to call provider-service HTTP API
• Remove Image field from Provider CRD and everywhere it's used in workflows (we don't need a different image per provider, just pass diff parameters to API call)

[chr12c]

To call the provider service api, we need to be able to send a URL encoded id from the workflow.
The delete api in particular expects an encoded id since they could involve /.

There’s 2 options:

• encode the URL from the workflow
• encode the URL from the Go code.

I think most would agree that leaking business logic into a yaml file isn't ideal since it's easier to control and unit test if it was within the Go code. To do this we can initially introduce another input parameter into the delete-related workflow templates; this won't break anything because nobody will be using it.

[alexgeorgousis]

To call the provider service api, we need to be able to send a URL encoded id from the workflow. The delete api in particular expects an encoded id since they could involve /.

There’s 2 options:

• encode the URL from the workflow
• encode the URL from the Go code.

I think most would agree that leaking business logic into a yaml file isn't ideal since it's easier to control and unit test if it was within the Go code. To do this we can initially introduce another input parameter into the delete-related workflow templates; this won't break anything because nobody will be using it.

What would the input parameter be? Also, is this another reason to keep the provider CLI and make the API request from there? Along with everything else we'll need to handle. Basically what I'm saying is:

I think most would agree that leaking business logic into a yaml file isn't ideal

Can we use this as an argument to have the entire API request logic happen inside the CLI?

[chr12c]

The parameter is the encoded resource-id.

I don't think this is a reason to keep the CLI just because the CLI can technically make an API request. This doesn't count as a reason for keeping it if the entire CLI can be removed if the template can make the call. The logic of encoding the id can be completely encapsulated in the construction of the workflow itself, i.e. the workflow factory. This doesn't seem too difficult.

What is the issue with introducing another input parameter?

Having less overhead is better:

workflow factory -> workflow cli -> api
workflow factory -> workflow http

Not to mention argo has support for http templates

[alexgeorgousis]

The parameter is the encoded resource-id.

Ahh yes that makes sense 👍

I don't think this is a reason to keep the CLI just because the CLI can technically make an API request. This doesn't count as a reason for keeping it if the entire CLI can be removed if the template can make the call. The logic of encoding the id can be completely encapsulated in the construction of the workflow itself, i.e. the workflow factory. This doesn't seem too difficult.

Yep, all I meant was, if making the call needs adding business logic to the WF, then we should consider moving it to the CLI.

What is the issue with introducing another input parameter?

No issue, sounds good 👍

Having less overhead is better:

workflow factory -> workflow cli -> api workflow factory -> workflow http

Not to mention argo has support for http templates

Yep agreed 💯 I don't actually know how much logic it takes, if it's very minimal then defo remove the CLI 👍

[grahamia]

The parameter is the encoded resource-id.

I don't think this is a reason to keep the CLI just because the CLI can technically make an API request. This doesn't count as a reason for keeping it if the entire CLI can be removed if the template can make the call. The logic of encoding the id can be completely encapsulated in the construction of the workflow itself, i.e. the workflow factory. This doesn't seem too difficult.

What is the issue with introducing another input parameter?

Having less overhead is better:

workflow factory -> workflow cli -> api workflow factory -> workflow http

Not to mention argo has support for http templates

Yes, we should be using the built in http workflow. It is quite standard that if you know the parameter you are sending could contain special characters then you should encode it to ensure no adverse side effects. This was what we discussed when we realised on testing the api call and added the decoder into the api that we would encode in the workflow. Agree it would be nice to unit test but think this will just need to be tested in the decoupled tests.