hi @raphaelblg ! a quick question: is it possible to write data from a DLT to a Kafka topic? Is this functionality implemented? I've seen that is a new create_sink() function, but I cannot find any information about.
Thanks Raphael! That's helpful. I'll look into the links.If I could ask you one more question, do you have any references or links to how upscaling and downscaling of the number of workers and cluster size is done for structured streaming? I have a u...
Hi @raphaelblg , thanks a lot for providing an elaborate answer. Do you happen to you, by any chance, of some solutions that developers use to track a consumer lag when streaming with Spark from a Kafka topic? It's a rather essential knowledge to hav...