Hi @DoTA  Thanks for the reply.  It looked like AI language at first, but I see you have some good content in here!  Hopefully it is based on real world experience.  I hate it when AI answers (like the ones from Genie) send us on wild goose chases.  The problem with AI is that it never says "no you can't do X".  Instead it hallucinates five or six options (X1 to X5) and makes you chase down all of them (and waste an entire week of time) before coming to your own independent conclusion that "no you can't do X".  lol.


For your #1 option, wouldn't that approach have been mentioned in the databricks docs for python as well?  It has a similar builder.  Perhaps python doesn't use librdkafka under the hood?  Your reply gives me some hope because I was pointed to librdkafka and to the following reference, for the things I can accomplish via this new interface in my old client runtime:
https://docs.confluent.io/platform/current/clients/librdkafka/html/md_CONFIGURATION.html

... I will google for SetProducerOption("sasl.oauthbearer.method", "oidc") and try to test those approaches asap!

For #2 option I do not think that suggestion is feasible, on a technical level, since my client runtime doesn't support a callback.  That is the main problem in the first place.  I think the approach still requires the callback as the starting point.

In regards to sidecars, I'm trying to avoid another client runtime as a sidecar.  It is ugly and adds a massive amount of latency that is possibly bigger than zerobus/kafka itself.  If my original client runtime can't just do something so simple (produce messages), then it is more primitive than python and should probably be killed/abandoned.  We waited many years for kafka support in this runtime, and it will probably be 10 or 100 more years for grRPC support.  It can't do AMQP messaging either.  All it has right now is a librdkafka wrapper (without support for a callback).