Observed JarFileReference "The reference counter cannot be negative, found: -1"

[jimbogithub]

Got the following error trace in production recently with Quarkus 3.14.3:

org.apache.kafka.clients.consumer.KafkaConsumer@74f8cbd4 failed: java.lang.IllegalStateException: The reference counter cannot be negative, found: -1
	at io.quarkus.bootstrap.runner.JarFileReference.release(JarFileReference.java:58)
	at io.quarkus.bootstrap.runner.JarFileReference.consumeSharedJarFile(JarFileReference.java:128)
	at io.quarkus.bootstrap.runner.JarFileReference.withJarFile(JarFileReference.java:96)
	at io.quarkus.bootstrap.runner.JarResource.getResourceData(JarResource.java:56)
	at io.quarkus.bootstrap.runner.RunnerClassLoader.loadClass(RunnerClassLoader.java:106)
	at io.quarkus.bootstrap.runner.RunnerClassLoader.loadClass(RunnerClassLoader.java:72)
	at org.apache.kafka.common.record.MemoryRecords.builder(MemoryRecords.java:591)
	at org.apache.kafka.common.record.MemoryRecords.builder(MemoryRecords.java:575)
	at org.apache.kafka.common.record.MemoryRecords.builder(MemoryRecords.java:523)
	at org.apache.kafka.clients.producer.internals.RecordAccumulator.recordsBuilder(RecordAccumulator.java:414)
	at org.apache.kafka.clients.producer.internals.RecordAccumulator.appendNewBatch(RecordAccumulator.java:398)
	at org.apache.kafka.clients.producer.internals.RecordAccumulator.append(RecordAccumulator.java:350)
	at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:1051)
	at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:962)
	...
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

The scenario is a Kafka Consumer/ Producer chain but I suspect that is irrelevant. Smells like a concurrency issue in io.quarkus.bootstrap.runner.JarFileReference. This was a one-off and I don't have a reproducer. I could file a Jira issue but I don't have much to go on here. Has anyone else ever seen this?

[jimbogithub]

P.S. In this scenario there are around 70 threads doing this same thing at the same time.

[cescoffier]

This issue has been fixed by #43309

[jimbogithub]

Thanks.