cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Delta table is not writing data read from kafka

Charbel
New Contributor II

Guys, could you help me? I'm reading 5 kafka threads through a list and saving the data in a Delta table The execution will be 1x a day, it seems that everything is working but I noticed that when I read the topic and it has no message, it still generates a directory with the delta table if on the same day I run it again and in this second run it reads some data in kafka even though I'm using ".outputMode("append")" it doesn't write anything to the existing delta table, now if I delete the checkpoint directory it works perfectly , what am I doing wrong ?

0693f000007OoRrAAK

1 REPLY 1

Kaniz
Community Manager
Community Manager

Hi @Charbel! My name is Kaniz, and I'm a technical moderator here. Great to meet you, and thanks for your question! Let's see if your peers on the Forum have an answer to your questions first. Or else I will follow up shortly with a response.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.