[avro] Can not serialize UUID to string #218

lukas-krecan · 2020-08-07T07:14:25Z

Due to this change in Jackson Avro it's not possible to serialize UUID to String.

public class AvroTest {
    private final String schemaStr = "{\n" +
            "  \"name\": \"Bean\",\n" +
            "  \"type\": \"record\",\n" +
            "  \"namespace\": \"example\",\n" +
            "  \"fields\": [\n" +
            "    {\n" +
            "      \"name\": \"uuid\",\n" +
            "      \"type\": \"string\"\n" +
            "    }\n" +
            "  ]\n" +
            "}";

    @Test
    void testAvro() throws JsonProcessingException {
        AvroMapper mapper = new AvroMapper();
        Schema schema =new Schema.Parser().parse(schemaStr);
        mapper.writer(new AvroSchema(schema)).writeValueAsBytes(new Bean());
    }

    private static class Bean {
        public UUID getUUID() {
            return UUID.randomUUID();
        }
    }
}

fails with

com.fasterxml.jackson.databind.JsonMappingException: class [B cannot be cast to class java.lang.CharSequence ([B and java.lang.CharSequence are in module java.base of loader 'bootstrap')

	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._wrapAsIOE(DefaultSerializerProvider.java:509)
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:482)
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.serializeValue(DefaultSerializerProvider.java:319)
	at com.fasterxml.jackson.databind.ObjectWriter$Prefetch.serialize(ObjectWriter.java:1513)
	at com.fasterxml.jackson.databind.ObjectWriter._configAndWriteValue(ObjectWriter.java:1215)
	at com.fasterxml.jackson.databind.ObjectWriter.writeValueAsBytes(ObjectWriter.java:1109)
	at AvroTest.testAvro(AvroTest.java:26)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:567)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:686)
	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
	at org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
	at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:212)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:208)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:137)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:71)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:135)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1507)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1507)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$5(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$7(NodeTestTask.java:125)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:135)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:123)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:122)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:80)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:248)
	at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$5(DefaultLauncher.java:211)
	at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:226)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:199)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:132)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:71)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:220)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:53)
Caused by: java.lang.ClassCastException: class [B cannot be cast to class java.lang.CharSequence ([B and java.lang.CharSequence are in module java.base of loader 'bootstrap')
	at org.apache.avro.generic.GenericDatumWriter.writeString(GenericDatumWriter.java:323)
	at org.apache.avro.generic.GenericDatumWriter.writeString(GenericDatumWriter.java:315)
	at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:150)
	at com.fasterxml.jackson.dataformat.avro.ser.NonBSGenericDatumWriter.write(NonBSGenericDatumWriter.java:123)
	at org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:206)
	at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:195)
	at org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:130)
	at com.fasterxml.jackson.dataformat.avro.ser.NonBSGenericDatumWriter.write(NonBSGenericDatumWriter.java:123)
	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
	at com.fasterxml.jackson.dataformat.avro.ser.RootContext.complete(RootContext.java:122)
	at com.fasterxml.jackson.dataformat.avro.AvroGenerator._complete(AvroGenerator.java:621)
	at com.fasterxml.jackson.dataformat.avro.AvroGenerator.writeEndObject(AvroGenerator.java:410)
	at com.fasterxml.jackson.databind.ser.BeanSerializer.serialize(BeanSerializer.java:180)
	at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider._serialize(DefaultSerializerProvider.java:480)

The text was updated successfully, but these errors were encountered:

cowtowncoder · 2020-08-10T21:12:03Z

Does

@JsonFormat(shape = JsonFormat.Shape.STRING)
public UUID getUUID() { }

work?

Or conversely use of "config overrides" for all instances of java.util.UUID, with something like

mapper.configOverride(UUID.class)
    .setFormat(JsonFormat.Value.forShape(JsonFormat.Shape.STRING))

lukas-krecan · 2020-08-11T18:01:09Z

@cowtowncoder Setting shape does not help

cowtowncoder · 2020-08-11T18:19:10Z

@lukas-krecan Ok. Thank you for verifying. Yes, looking at UUIDSerializer, it currently only considers whether backend can write binary natively (which formerly Avro backend claimed it couldn't), and bases decision on that.

I'll create a ticket for jackson-databind to make UUIDSerializer respect JsonFormat.Shape overrides, to allow resolving this problem.

cowtowncoder · 2020-08-11T18:25:20Z

Created FasterXML/jackson-databind#2815 for tracing fix for this issue.

In hindsight I should have made sure shape overrides were working when finishing 2.11.

cowtowncoder · 2020-11-28T01:08:25Z

So; fixed in 2.11.3 for jackson-databind. To force serialization as String with Avro, use either per-property annotation:

@JsonFormat(shape = JsonFormat.Shape.STRING)
UUID id;

or configure mapper with config override:

        ObjectMapper m = new AvroMapper();
        m.configOverride(UUID.class)
            .setFormat(JsonFormat.Value.forShape(JsonFormat.Shape.STRING));

lukas-krecan · 2020-11-28T07:49:57Z

Thanks a lot

cowtowncoder mentioned this issue Aug 11, 2020

Add JsonFormat.Shape awareness for UUID serialization (UUIDSerializer) FasterXML/jackson-databind#2815

Closed

cowtowncoder closed this as completed Nov 28, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[avro] Can not serialize UUID to string #218

[avro] Can not serialize UUID to string #218

lukas-krecan commented Aug 7, 2020 •

edited

Loading

cowtowncoder commented Aug 10, 2020

lukas-krecan commented Aug 11, 2020

cowtowncoder commented Aug 11, 2020

cowtowncoder commented Aug 11, 2020

cowtowncoder commented Nov 28, 2020

lukas-krecan commented Nov 28, 2020

[avro] Can not serialize UUID to string #218

[avro] Can not serialize UUID to string #218

Comments

lukas-krecan commented Aug 7, 2020 • edited Loading

cowtowncoder commented Aug 10, 2020

lukas-krecan commented Aug 11, 2020

cowtowncoder commented Aug 11, 2020

cowtowncoder commented Aug 11, 2020

cowtowncoder commented Nov 28, 2020

lukas-krecan commented Nov 28, 2020

lukas-krecan commented Aug 7, 2020 •

edited

Loading