The group already exists when creating a new artifact multiple repeatably.

[stijn-vanpoucke-cegeka]

Description

For an application producing multiple messages using Kafka and Kafka-Streams we're using an in-memory testcontainer image in our integration-tests. We noticed that when we do multiple test runs that the RegistryClientImpl.createArtifact method sometimes throws a RestClientException exception with message The group [our groupname] already exists and then fails. Possibly due timing issues inside the registry when calling it multiple times from multiple threads on a short notice.

Registry
Version: 2.6.1-final
Persistence type: in-memory

Environment

Testcontainers / Docker Desktop used inside a junit test. The container get's recreated on every test run. This is the way how we created the repeatability causing the error in exceptional cases.

Steps to Reproduce

Using Java:

• Add the following maven or gradle dependency io.apicurio:apicurio-registry-serdes-avro-serde:2.6.1.Final
• AUTO_REGISTER_ARTIFACT should be set to TRUE
• AUTO_REGISTER_ARTIFACT_IF_EXISTS should be set to RETURN_OR_UPDATE
• Setup an integration test that starts a new container for every test run
• Serialize the same message repeatably from multiple threads (in our case using Kafka-Streams)

Expected vs Actual Behaviour

Excepted: The schema get's created every time over and over again without any issues.
Actual: Flackyness where The group [our groupname] already exists RestClientException gets thrown in a minority of the test runs.

Logs

io.apicurio.registry.rest.client.exception.RestClientException: The group 'app......keys' already exists. at app//io.apicurio.registry.rest.client.impl.ErrorHandler.handleErrorResponse(ErrorHandler.java:64) at app//io.apicurio.rest.client.handler.BodyHandler.lambda$toSupplierOfType$1(BodyHandler.java:55) at app//io.apicurio.rest.client.JdkHttpClient.sendRequest(JdkHttpClient.java:204) at app//io.apicurio.registry.rest.client.impl.RegistryClientImpl.createArtifact(RegistryClientImpl.java:311) at app//io.apicurio.registry.rest.client.RegistryClient.createArtifact(RegistryClient.java:147) at app//io.apicurio.registry.resolver.DefaultSchemaResolver.lambda$handleAutoCreateArtifact$3(DefaultSchemaResolver.java:278) at app//io.apicurio.registry.resolver.ERCache.lambda$getValue$0(ERCache.java:201) at app//io.apicurio.registry.resolver.ERCache.retry(ERCache.java:254) at app//io.apicurio.registry.resolver.ERCache.getValue(ERCache.java:200) at app//io.apicurio.registry.resolver.ERCache.getByContent(ERCache.java:175) at app//io.apicurio.registry.resolver.DefaultSchemaResolver.handleAutoCreateArtifact(DefaultSchemaResolver.java:276) at app//io.apicurio.registry.resolver.DefaultSchemaResolver.getSchemaFromRegistry(DefaultSchemaResolver.java:123) at app//io.apicurio.registry.resolver.DefaultSchemaResolver.resolveSchema(DefaultSchemaResolver.java:91) at app//io.apicurio.registry.serde.AbstractKafkaSerializer.serialize(AbstractKafkaSerializer.java:83) at app//io.apicurio.registry.serde.AbstractKafkaSerializer.serialize(AbstractKafkaSerializer.java:70) at app//org.apache.kafka.streams.state.StateSerdes.rawKey(StateSerdes.java:174) at app//org.apache.kafka.streams.state.internals.MeteredKeyValueStore.keyBytes(MeteredKeyValueStore.java:322) at app//org.apache.kafka.streams.state.internals.MeteredKeyValueStore.lambda$get$2(MeteredKeyValueStore.java:209) at app//org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl.maybeMeasureLatency(StreamsMetricsImpl.java:824) at app//org.apache.kafka.streams.state.internals.MeteredKeyValueStore.get(MeteredKeyValueStore.java:209) at app//org.apache.kafka.streams.state.internals.ReadOnlyKeyValueStoreFacade.get(ReadOnlyKeyValueStoreFacade.java:35) at app//org.apache.kafka.streams.state.internals.CompositeReadOnlyKeyValueStore.get(CompositeReadOnlyKeyValueStore.java:56

[carlesarnal]

@EricWittmann I think this has been addressed by #5169, no?

[EricWittmann]

Yes it should be fixed now.