cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

What is the best way to decode protobuf using pyspark?

User16868770416
Contributor

I am using spark structured streaming to read a protobuf encoded message from the event hub. We use a lot of Delta tables, but there isn't a simple way to integrate this.

We are currently using K-SQL to transform into avro on the fly and then use Databricks to read the Avro file. Is there a better way of doing this?

1 REPLY 1

jose_gonzalez
Moderator
Moderator

hi @Will Block​ ,

I think there is a related question being asked in the past. I think it was this one

I found this library, I hope it helps.

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.