Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(operator): Add support for properly configuring the CORS allowed origins header #5906

Merged
merged 3 commits into from
Jan 24, 2025
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -10,4 +10,6 @@ public class EnvironmentVariables {
public static final String APICURIO_REST_DELETION_ARTIFACT_ENABLED = "APICURIO_REST_DELETION_ARTIFACT_ENABLED";
public static final String APICURIO_REST_DELETION_GROUP_ENABLED = "APICURIO_REST_DELETION_GROUP_ENABLED";

public static final String APICURIO_REST_MUTABILITY_ARTIFACT_VERSION_CONTENT_ENABLED = "APICURIO_REST_MUTABILITY_ARTIFACT-VERSION-CONTENT_ENABLED";

}
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
package io.apicurio.registry.operator.feat;

import io.apicurio.registry.operator.EnvironmentVariables;
import io.apicurio.registry.operator.api.v1.ApicurioRegistry3;
import io.apicurio.registry.operator.api.v1.ApicurioRegistry3Spec;
import io.apicurio.registry.operator.api.v1.spec.ComponentSpec;
import io.apicurio.registry.operator.resource.ResourceFactory;
import io.apicurio.registry.operator.utils.IngressUtils;
import io.fabric8.kubernetes.api.model.EnvVar;
import io.fabric8.kubernetes.api.model.EnvVarBuilder;

import java.util.Arrays;
import java.util.LinkedHashMap;
import java.util.Optional;
import java.util.Set;
import java.util.TreeSet;

/**
* Helper class used to handle CORS related configuration.
*/
public class Cors {
/**
* Configure the QUARKUS_HTTP_CORS_ORIGINS environment variable with the following:
* <ul>
* <li>Add the ingress host</li>
* <li>Include anything specifically configured for QUARKUS_HTTP_CORS_ORIGINS in the "env" section</li>
* </ul>
*
* @param primary
* @param envVars
*/
public static void configureAllowedOrigins(ApicurioRegistry3 primary,
LinkedHashMap<String, EnvVar> envVars) {
TreeSet<String> allowedOrigins = new TreeSet<>();

// If the QUARKUS_HTTP_CORS_ORIGINS env var is configured in the "env" section of the CR,
// then make sure to add those configured values to the set of allowed origins we want to
// configure.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a valid functionality, but I think we should follow a principle regarding configuration in the env section:

The user should be able to completely override any env. variable set by the operator. i.e. we should not touch variables set by the user, and they should override the ones set by the operator.

The advantage of this is that (together with PTS), there is a workaround for users to handle most situations even if there is a bug in the operator (e.g. setting an incorrect env. var value), or there is some circumstance we were not able to predict. This is also simple to explain to the users.

Disadvantage might be a slight decrease in user experience, where the operator could do more things automatically, but I don't think this would affect a lot of feature.

WDYT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: The method io.apicurio.registry.operator.resource.app.AppDeploymentResource#addEnvVar is designed to do this, and we should consolidate a way env. vars are added to a handful of methods, maybe in EnvironmentVariables.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean that the value configured in the env section should take precedent (override) the rest of the logic? I can see why that makes sense. Please confirm and I'll change it.

Optional.ofNullable(primary.getSpec()).map(ApicurioRegistry3Spec::getApp).map(ComponentSpec::getEnv)
.ifPresent(env -> {
env.stream().filter(
envVar -> envVar.getName().equals(EnvironmentVariables.QUARKUS_HTTP_CORS_ORIGINS))
.forEach(envVar -> {
Optional.ofNullable(envVar.getValue()).ifPresent(envVarValue -> {
Arrays.stream(envVarValue.split(",")).forEach(allowedOrigins::add);
});
});
});

// If there is a configured Ingress host for the UI or the Studio UI, add them to the allowed origins.
Set.of(ResourceFactory.COMPONENT_UI, ResourceFactory.COMPONENT_STUDIO_UI).forEach(component -> {
String host = IngressUtils.getConfiguredHost(component, primary);
if (host != null) {
allowedOrigins.add("http://" + host);
allowedOrigins.add("https://" + host);
}
});

if (allowedOrigins.isEmpty()) {
allowedOrigins.add("*");
}

// Join the values in allowedOrigins into a String and set it as the new value of the env var.
String envVarValue = String.join(",", allowedOrigins);
envVars.put(EnvironmentVariables.QUARKUS_HTTP_CORS_ORIGINS, new EnvVarBuilder()
.withName(EnvironmentVariables.QUARKUS_HTTP_CORS_ORIGINS).withValue(envVarValue).build());
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
import io.apicurio.registry.operator.api.v1.spec.AppFeaturesSpec;
import io.apicurio.registry.operator.api.v1.spec.AppSpec;
import io.apicurio.registry.operator.api.v1.spec.StorageSpec;
import io.apicurio.registry.operator.feat.Cors;
import io.apicurio.registry.operator.feat.KafkaSql;
import io.apicurio.registry.operator.feat.PostgresSql;
import io.fabric8.kubernetes.api.model.Container;
Expand Down Expand Up @@ -60,7 +61,6 @@ protected Deployment desired(ApicurioRegistry3 primary, Context<ApicurioRegistry
// spotless:off
addEnvVar(envVars, new EnvVarBuilder().withName(EnvironmentVariables.QUARKUS_PROFILE).withValue("prod").build());
addEnvVar(envVars, new EnvVarBuilder().withName(EnvironmentVariables.QUARKUS_HTTP_ACCESS_LOG_ENABLED).withValue("true").build());
addEnvVar(envVars, new EnvVarBuilder().withName(EnvironmentVariables.QUARKUS_HTTP_CORS_ORIGINS).withValue("*").build());

// Enable deletes if configured in the CR
boolean allowDeletes = Optional.ofNullable(primary.getSpec().getApp())
Expand All @@ -72,18 +72,23 @@ protected Deployment desired(ApicurioRegistry3 primary, Context<ApicurioRegistry
addEnvVar(envVars, new EnvVarBuilder().withName(EnvironmentVariables.APICURIO_REST_DELETION_ARTIFACT_ENABLED).withValue("true").build());
addEnvVar(envVars, new EnvVarBuilder().withName(EnvironmentVariables.APICURIO_REST_DELETION_GROUP_ENABLED).withValue("true").build());
}
// spotless:on

// This is enabled only if Studio is deployed. It is based on Service in case a custom Ingress is
// used.
// Configure the CORS_ALLOWED_ORIGINS env var based on the ingress host
Cors.configureAllowedOrigins(primary, envVars);

// Enable the "mutability" feature in Registry, but only if Studio is deployed. It is based on Service
// in case a custom Ingress is used.
var sOpt = context.getSecondaryResource(STUDIO_UI_SERVICE_KEY.getKlass(),
STUDIO_UI_SERVICE_KEY.getDiscriminator());
sOpt.ifPresent(s -> {
addEnvVar(envVars,
new EnvVarBuilder().withName("APICURIO_REST_MUTABILITY_ARTIFACT-VERSION-CONTENT_ENABLED")
new EnvVarBuilder().withName(EnvironmentVariables.APICURIO_REST_MUTABILITY_ARTIFACT_VERSION_CONTENT_ENABLED)
.withValue("true").build());
});

// spotless:on

// Configure the storage (Postgresql or KafkaSql).
ofNullable(primary.getSpec()).map(ApicurioRegistry3Spec::getApp).map(AppSpec::getStorage)
.map(StorageSpec::getType).ifPresent(storageType -> {
switch (storageType) {
Expand All @@ -92,6 +97,7 @@ protected Deployment desired(ApicurioRegistry3 primary, Context<ApicurioRegistry
}
});

// Set the ENV VARs on the deployment's container spec.
var container = getContainerFromDeployment(d, REGISTRY_APP_CONTAINER_NAME);
container.setEnv(envVars.values().stream().toList());

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -28,21 +28,39 @@ public final class IngressUtils {
private IngressUtils() {
}

public static String getHost(String component, ApicurioRegistry3 p) {
/**
* Get the host configured in the ingress. If no host is configured in the ingress then a null is
* returned.
*
* @param component
* @param primary
*/
public static String getConfiguredHost(String component, ApicurioRegistry3 primary) {
String host = switch (component) {
case COMPONENT_APP -> ofNullable(p.getSpec()).map(ApicurioRegistry3Spec::getApp)
case COMPONENT_APP -> ofNullable(primary.getSpec()).map(ApicurioRegistry3Spec::getApp)
EricWittmann marked this conversation as resolved.
Show resolved Hide resolved
.map(AppSpec::getIngress).map(IngressSpec::getHost).filter(h -> !isBlank(h)).orElse(null);
case COMPONENT_UI -> ofNullable(p.getSpec()).map(ApicurioRegistry3Spec::getUi)
case COMPONENT_UI -> ofNullable(primary.getSpec()).map(ApicurioRegistry3Spec::getUi)
.map(UiSpec::getIngress).map(IngressSpec::getHost).filter(h -> !isBlank(h)).orElse(null);
case COMPONENT_STUDIO_UI ->
ofNullable(p.getSpec()).map(ApicurioRegistry3Spec::getStudioUi).map(StudioUiSpec::getIngress)
.map(IngressSpec::getHost).filter(h -> !isBlank(h)).orElse(null);
case COMPONENT_STUDIO_UI -> ofNullable(primary.getSpec()).map(ApicurioRegistry3Spec::getStudioUi)
.map(StudioUiSpec::getIngress).map(IngressSpec::getHost).filter(h -> !isBlank(h))
.orElse(null);
default -> throw new OperatorException("Unexpected value: " + component);
};
return host;
}

/**
* Get the host for an ingress. If not configured, a default value is returned.
*
* @param component
* @param primary
*/
public static String getHost(String component, ApicurioRegistry3 primary) {
String host = getConfiguredHost(component, primary);
if (host == null) {
// TODO: This is not used because of the current activation conditions.
host = "%s-%s.%s%s".formatted(p.getMetadata().getName(), component,
p.getMetadata().getNamespace(), Configuration.getDefaultBaseHost());
host = "%s-%s.%s%s".formatted(primary.getMetadata().getName(), component,
primary.getMetadata().getNamespace(), Configuration.getDefaultBaseHost());
}
log.debug("Host for component {} is {}", component, host);
return host;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,14 @@ public class Mapper {
YAML_MAPPER.configure(FAIL_ON_UNKNOWN_PROPERTIES, true);
}

public static <T> T deserialize(String data, Class<T> klass) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar functionality is already in io.apicurio.registry.operator.resource.ResourceFactory#deserialize. Can we a single place for this functionality? I don't have a preference for where to put it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same answer as above.

try {
return YAML_MAPPER.readValue(data, klass);
} catch (JsonProcessingException ex) {
throw new OperatorException("Could not deserialize resource.", ex);
}
}

public static String toYAML(Object value) {
try {
return YAML_MAPPER.writeValueAsString(value);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,6 @@
import io.fabric8.kubernetes.api.model.EnvVar;
import io.quarkus.test.junit.QuarkusTest;
import org.junit.jupiter.api.Test;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import static io.apicurio.registry.operator.api.v1.ContainerNames.REGISTRY_APP_CONTAINER_NAME;
import static io.apicurio.registry.operator.resource.app.AppDeploymentResource.getContainerFromDeployment;
Expand All @@ -17,8 +15,6 @@
@QuarkusTest
public class AppFeaturesITTest extends ITBase {

private static final Logger log = LoggerFactory.getLogger(AppFeaturesITTest.class);

@Test
void testAllowDeletesTrue() {
ApicurioRegistry3 registry = ResourceFactory
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package io.apicurio.registry.operator.it;

import io.apicurio.registry.operator.EnvironmentVariables;
import io.apicurio.registry.operator.api.v1.ApicurioRegistry3;
import io.apicurio.registry.operator.resource.ResourceFactory;
import io.fabric8.kubernetes.api.model.Container;
Expand All @@ -15,9 +16,11 @@

import java.net.URI;

import static io.apicurio.registry.operator.api.v1.ContainerNames.REGISTRY_APP_CONTAINER_NAME;
import static io.apicurio.registry.operator.api.v1.ContainerNames.REGISTRY_UI_CONTAINER_NAME;
import static io.apicurio.registry.operator.resource.ResourceFactory.COMPONENT_APP;
import static io.apicurio.registry.operator.resource.ResourceFactory.COMPONENT_UI;
import static io.apicurio.registry.operator.resource.app.AppDeploymentResource.getContainerFromDeployment;
import static io.restassured.RestAssured.given;
import static org.assertj.core.api.Assertions.assertThat;
import static org.awaitility.Awaitility.await;
Expand Down Expand Up @@ -70,6 +73,18 @@ void smoke() {
.get(0).getHost()).isEqualTo(registry.getSpec().getUi().getIngress().getHost());
return true;
});

// Check CORS allowed origins is set on the app, with the value based on the UI ingress host
String uiIngressHost = client.network().v1().ingresses().inNamespace(namespace)
.withName(registry.getMetadata().getName() + "-ui-ingress").get().getSpec().getRules().get(0)
.getHost();
String corsOriginsExpectedValue = "http://" + uiIngressHost + "," + "https://" + uiIngressHost;
var appEnv = getContainerFromDeployment(
client.apps().deployments().inNamespace(namespace)
.withName(registry.getMetadata().getName() + "-app-deployment").get(),
REGISTRY_APP_CONTAINER_NAME).getEnv();
assertThat(appEnv).map(ev -> ev.getName() + "=" + ev.getValue())
.contains(EnvironmentVariables.QUARKUS_HTTP_CORS_ORIGINS + "=" + corsOriginsExpectedValue);
}

@Test
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
package io.apicurio.registry.operator.unit;

import io.apicurio.registry.operator.EnvironmentVariables;
import io.apicurio.registry.operator.OperatorException;
import io.apicurio.registry.operator.api.v1.ApicurioRegistry3;
import io.apicurio.registry.operator.feat.Cors;
import io.apicurio.registry.operator.utils.Mapper;
import io.fabric8.kubernetes.api.model.EnvVar;
import org.assertj.core.api.Assertions;
import org.junit.jupiter.api.Test;

import java.nio.charset.Charset;
import java.util.LinkedHashMap;
import java.util.Set;

public class CorsTest {

private static final String DEFAULT = """
apiVersion: registry.apicur.io/v1
kind: ApicurioRegistry3
metadata:
name: simple
spec: {}
""";

private static final String WITH_INGRESS = """
apiVersion: registry.apicur.io/v1
kind: ApicurioRegistry3
metadata:
name: simple
spec:
app:
ingress:
host: simple-app.apps.cluster.example
ui:
ingress:
host: simple-ui.apps.cluster.example
""";

private static final String WITH_ENV_VAR = """
apiVersion: registry.apicur.io/v1
kind: ApicurioRegistry3
metadata:
name: simple
spec:
app:
env:
- name: QUARKUS_HTTP_CORS_ORIGINS
value: https://ui.example.org
""";

private static final String WITH_ENV_VAR_AND_INGRESS = """
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should use the files in operator/controller/src/test/resources/k8s/examples as much as possible, even if they are simple (or slightly edited in the test). This would provide resources we can use in the docs and where we can point users to. The examples will end up in the "install examples" release artifact.

(BTW similarly I have an aspiration to eventually do the same thing for the registry examples and run them as tests).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was having trouble getting those resources to load in a unit test vs an IT test (the context classloader is different). I didn't want to duplicate the resource loading, but I can of course. I'll go ahead and move these into k8s/exmaples.

apiVersion: registry.apicur.io/v1
kind: ApicurioRegistry3
metadata:
name: simple
spec:
app:
ingress:
host: simple-app.apps.cluster.example
env:
- name: QUARKUS_HTTP_CORS_ORIGINS
value: https://ui.example.org
ui:
ingress:
host: simple-ui.apps.cluster.example
""";

@Test
public void testConfigureAllowedOrigins() throws Exception {
doTestAllowedOrigins(DEFAULT, "*");
doTestAllowedOrigins(WITH_INGRESS, "http://simple-ui.apps.cluster.example",
"https://simple-ui.apps.cluster.example");
doTestAllowedOrigins(WITH_ENV_VAR, "https://ui.example.org");
doTestAllowedOrigins(WITH_ENV_VAR_AND_INGRESS, "http://simple-ui.apps.cluster.example",
"https://simple-ui.apps.cluster.example", "https://ui.example.org");
}

private void doTestAllowedOrigins(String cr, String... values) {
ApicurioRegistry3 registry = Mapper.deserialize(cr, ApicurioRegistry3.class);

LinkedHashMap<String, EnvVar> envVars = new LinkedHashMap<>();
Cors.configureAllowedOrigins(registry, envVars);
Assertions.assertThat(envVars.keySet()).contains(EnvironmentVariables.QUARKUS_HTTP_CORS_ORIGINS);
String allowedOriginsValue = envVars.get(EnvironmentVariables.QUARKUS_HTTP_CORS_ORIGINS).getValue();
Set<String> allowedOrigins = Set.of(allowedOriginsValue.split(","));
Assertions.assertThat(allowedOrigins).containsExactlyInAnyOrder(values);
}

public static String load(String path) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar functionality is already in io.apicurio.registry.operator.resource.ResourceFactory#load. Can we a single place for this functionality? I don't have a preference for where to put it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The resource loading in ResourceFactory didn't work from a unit test, I assume due to the difference in thread context classloader. I can consolidate as long as you're OK with multiple variants of load() for the different use cases. For the unit test I will need to pass a classloader to load() so it can properly load the resource.

try (var stream = Thread.currentThread().getContextClassLoader().getResourceAsStream(path)) {
return new String(stream.readAllBytes(), Charset.defaultCharset());
} catch (Exception ex) {
throw new OperatorException("Could not read resource: " + path, ex);
}
}

}
Loading