Setting up protobuf decoding with a custom proto file

[apatnaik14]

Hello everyone!

I was running into some issues with setting up trino with a custom protobuf file and parsing messages from kafka. The specific exceptions I have seen so far are below:

trino | java.lang.IllegalArgumentException: resource /usr/lib/trino/plugin/kafka/file2.proto relative to io.trino.decoder.protobuf.ProtobufUtils not found. trino | at com.google.common.base.Preconditions.checkArgument(Preconditions.java:445) trino | at com.google.common.io.Resources.getResource(Resources.java:209) trino | at io.trino.decoder.protobuf.ProtobufUtils.getProtoFile(ProtobufUtils.java:134) trino | at io.trino.decoder.protobuf.ProtobufUtils.getFileDescriptor(ProtobufUtils.java:93) trino | at io.trino.decoder.protobuf.ProtobufUtils.getFileDescriptor(ProtobufUtils.java:82) trino | at io.trino.decoder.protobuf.ProtobufUtils.getFileDescriptor(ProtobufUtils.java:75) trino | at io.trino.decoder.protobuf.FixedSchemaDynamicMessageProvider$Factory.create(FixedSchemaDynamicMessageProvider.java:58) trino | at io.trino.decoder.protobuf.ProtobufRowDecoderFactory.create(ProtobufRowDecoderFactory.java:49) trino | at io.trino.decoder.DispatchingRowDecoderFactory.create(DispatchingRowDecoderFactory.java:37) trino | at io.trino.plugin.kafka.KafkaRecordSetProvider.getRecordSet(KafkaRecordSetProvider.java:72) trino | at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorRecordSetProvider.getRecordSet(ClassLoaderSafeConnectorRecordSetProvider.java:48) trino | at io.trino.split.RecordPageSourceProvider.createPageSource(RecordPageSourceProvider.java:50) trino | at io.trino.split.PageSourceManager.createPageSource(PageSourceManager.java:61) trino | at io.trino.operator.TableScanOperator.getOutput(TableScanOperator.java:263) trino | at io.trino.operator.Driver.processInternal(Driver.java:395) trino | at io.trino.operator.Driver.lambda$process$8(Driver.java:298) trino | at io.trino.operator.Driver.tryWithLock(Driver.java:694) trino | at io.trino.operator.Driver.process(Driver.java:290) trino | at io.trino.operator.Driver.processForDuration(Driver.java:261) trino | at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:887) trino | at io.trino.execution.executor.timesharing.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:187) trino | at io.trino.execution.executor.timesharing.TimeSharingTaskExecutor$TaskRunner.run(TimeSharingTaskExecutor.java:565) trino | at io.trino.$gen.Trino_434____20231208_170529_2.run(Unknown Source) trino | at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) trino | at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) trino | at java.base/java.lang.Thread.run(Thread.java:1583) trino |

I was hoping to ask if anyone else has seen an error like above while configuring the trino kafka operator?

I am using a custom docker image for my local testing

`FROM trinodb/trino
USER trino:trino

COPY --chown=trino:trino protobuf-java-3.16.3.jar /usr/lib/trino/plugin/kafka/protobuf-java-3.16.3.jar
COPY --chown=trino:trino file1.proto /usr/lib/trino/plugin/kafka/file1.proto
COPY --chown=trino:trino file2.proto /usr/lib/trino/plugin/kafka/file2.proto`

I have tried passing the full path in the data schema, as well as tweaking the file1.proto file to see if the import of file2 in it is causing any issues.