Hi Everyone,

Hope everyone is enjoying their holiday time.

I have spark streaming job (copy activity from location A to location B) and a Query Listener and attached to the streaming job.

As mentioned over the attachments I can see numInputRows gets logged to driver logs. But I can't write the finaldf to delta table and even can't log the dataframe to driver logs when I do "show".

I remember the same piece code used to work on earlier versions may be at spark 3.* which was an year back.

Please let me know if I am doing something wrong or suggest me if changes required.

Thanks

Harish Koduru