[KafkaSQL] JdbcSQLIntegrityConstraintViolationException when creating artifacts concurrently

[EricWittmann]

Description

Registry Version: 2.6.x
Persistence type: KafkaSQL

When creating artifacts with the same content in parallel, this exception is thrown:

Exception in thread "pool-2-thread-1" io.apicurio.registry.rest.client.exception.RestClientException: io.apicurio.registry.storage.impl.sql.jdb.RuntimeSqlException: org.h2.jdbc.JdbcSQLIntegrityConstraintViolationException: Unique index or primary key violation: "PUBLIC.UNQ_CONTENT_1_INDEX_6 ON PUBLIC.CONTENT(TENANTID NULLS FIRST, CONTENTHASH NULLS FIRST) VALUES ( /* key:971 */ '_', 'c442c8ba5daa1ea8d39793d8e1eabbd819f6b258aaa5dc16a695edce95c5f076')"; SQL statement:
INSERT INTO content (tenantId, contentId, canonicalHash, contentHash, content, artifactreferences) VALUES (?, ?, ?, ?, ?, ?) [23505-220]
        at io.apicurio.registry.rest.client.impl.ErrorHandler.handleErrorResponse(ErrorHandler.java:64)
        at io.apicurio.rest.client.handler.BodyHandler.lambda$toSupplierOfType$1(BodyHandler.java:54)
        at io.apicurio.rest.client.JdkHttpClient.sendRequest(JdkHttpClient.java:204)
        at io.apicurio.registry.rest.client.impl.RegistryClientImpl.createArtifact(RegistryClientImpl.java:311)
        at io.apicurio.registry.rest.client.RegistryClient.createArtifact(RegistryClient.java:147)
        at io.apicurio.registry.resolver.DefaultSchemaResolver.lambda$handleAutoCreateArtifact$3(DefaultSchemaResolver.java:270)
        at io.apicurio.registry.resolver.ERCache.lambda$getValue$0(ERCache.java:156)
        at io.apicurio.registry.resolver.ERCache.retry(ERCache.java:197)
        at io.apicurio.registry.resolver.ERCache.getValue(ERCache.java:155)
        at io.apicurio.registry.resolver.ERCache.getByContent(ERCache.java:130)
        at io.apicurio.registry.resolver.DefaultSchemaResolver.handleAutoCreateArtifact(DefaultSchemaResolver.java:268)
        at io.apicurio.registry.resolver.DefaultSchemaResolver.handleArtifactReferences(DefaultSchemaResolver.java:145)
        at io.apicurio.registry.resolver.DefaultSchemaResolver.handleArtifactReferences(DefaultSchemaResolver.java:142)
        at io.apicurio.registry.resolver.DefaultSchemaResolver.getSchemaFromRegistry(DefaultSchemaResolver.java:116)
        at io.apicurio.registry.resolver.DefaultSchemaResolver.resolveSchema(DefaultSchemaResolver.java:90)
        at io.apicurio.registry.serde.AbstractKafkaSerializer.serialize(AbstractKafkaSerializer.java:83)
        at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:954)
        at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:914)
        at brave.kafka.clients.TracingProducer.send(TracingProducer.java:129)
        at org.springframework.kafka.core.DefaultKafkaProducerFactory$CloseSafeProducer.send(DefaultKafkaProducerFactory.java:1087)
        at org.springframework.kafka.core.KafkaTemplate.doSend(KafkaTemplate.java:655)
        at org.springframework.kafka.core.KafkaTemplate.send(KafkaTemplate.java:429)
        at cz.ifortuna.eps.beter.importer.Feed.processTimetableMessage(Feed.java:361)
        at cz.ifortuna.eps.beter.importer.Feed.lambda$onUpdateTimeTable$1(Feed.java:294)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:833)

Steps to Reproduce

Note: reproducer shell script attached.

[EricWittmann]

# Simple reproducer script for Apicurio registry "Unique index violation" issue
################################################################################ 

SCHEMA_REGISTRY_URL=http://localhost:8080
GROUP_ID=com.experimental.some.test.message

# runs several curl commands in parallel (note '&' at the end of the command) to create a simple 'TestMessage' in the registry
for i in {1..5}
do  
	curl --insecure -H 'X-Registry-Artifactid: TestMessage' -H 'X-Registry-Artifacttype: AVRO' -X POST $SCHEMA_REGISTRY_URL/apis/registry/v2/groups/$GROUP_ID/artifacts -d '{"type":"record","name":"TestMessage","namespace":"com.experimental.some.test.message","doc":"Test message","fields":[{"name":"created","type":{"type":"long","logicalType":"timestamp-millis"},"doc":"Test message time created"},{"name":"content","type":{"type":"string"},"doc":"Test message content"}]}' &
done

# Clean "com.experimental.some.test.message" group from the registry if needed
# curl --insecure -X DELETE $SCHEMA_REGISTRY_URL/apis/registry/v2/groups/$GROUP_ID/artifacts'

wait

[EricWittmann]

Fixed in:

• [3.x] Concurrent artifact creation test & fixes #5180
• [2.6.x] Concurrent artifact creation test & fixes #5169