protobuf deserialization warning for string keys

[solaristhesun]

The README file states "If, for example, keys are not in Protobuf format, key-message-type can be omitted, the same for value-message-type.". That is what I did because I am using protobuf deserialization for the values and strings for the keys. Why am I seeing the warning "Protobuf deserialization is configured for topic [...], but message type is not specified neither for a key, nor for a value" on every message?

My configuration looks like this
deserialization:
protobuf:
descriptors-folder: "/app/protobuf_desc"
topics-mapping:
- topic-regex: "topic1|topic2"
descriptor-file: "kafka.desc"
value-message-type: "DataTypeX"

[solaristhesun]

The deserialization works correctly. It's just the warning message that bothers me.

[tchiotludo]

@xakassi, any idea ?

[xakassi]

Hi, guys!
It is really strange. So the message is exactly "Protobuf deserialization is configured for topic [...], but message type is not specified neither for a key, nor for a value"? Topic name is [...]? Weird.

It seems like this strange topic matches the regex here:
https://github.com/tchiotludo/akhq/blob/dev/src/main/java/org/akhq/utils/ProtobufToJsonDeserializer.java#L115

But actually no rules are configured for it, so the warning appears:
https://github.com/tchiotludo/akhq/blob/dev/src/main/java/org/akhq/utils/ProtobufToJsonDeserializer.java#L125

Actually we do not have the latest AKHQ version with Protobuf yet in our company. But we are planning to get it really soon. Next week or 2 weeks, I suppose. So I will check it on our real data and our customers configs. I will let you know!

[solaristhesun]

It is really strange. So the message is exactly "Protobuf deserialization is configured for topic [...], but message type is not specified neither for a key, nor for a value"? Topic name is [...]? Weird.

Yes.

[xakassi]

@solaristhesun So the topic name is actually not "...", but some real topic name "bga-data" (as on the screenshot)? Does this topic name match your regex ("topic1|topic2" is just an example, right, but actual value is different)?

- topic-regex: "topic1|topic2"
descriptor-file: "kafka.desc"
value-message-type: "DataTypeX"

Could you, please, provide full AKHQ config? It would be better to see full and real (as much as it possible) values of the config to figure out what can be wrong.

[solaristhesun]

Yes, it was an example. I tried to anonymize it in order to not leak any sensitive information :). But anyways, here es the full configuration:

  AKHQ_CONFIGURATION: |
    akhq:
      connections:
        kafka-cluster:
          properties:      
            bootstrap.servers: "kafka:9092"
          deserialization:
            protobuf:
              descriptors-folder: "/app/protobuf_desc"
              topics-mapping:
                - topic-regex: "bga-data|oxy-data"
                  descriptor-file: "kafka.desc"
                  value-message-type: "MValueSet"
                - topic-regex: "patients"
                  descriptor-file: "kafka.desc"
                  value-message-type: "Patient"       

[xakassi]

I reproduced this bug! Really it is a little bug in my code. I will fix it! PR will be soon :)

And yes, it is harmless, just an annoying message on UI.

[solaristhesun]

@xakassi Thank you very much!

[xakassi]

Hi, @tchiotludo !
Please, check and merge my PR, when you are free, it is really minor change, it was such a stupid mistake in my code 😮