cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

What is the best way to decode protobuf using pyspark?

WillBlock
Contributor

I am using spark structured streaming to read a protobuf encoded message from the event hub. We use a lot of Delta tables, but there isn't a simple way to integrate this.

We are currently using K-SQL to transform into avro on the fly and then use Databricks to read the Avro file. Is there a better way of doing this?

1 REPLY 1

jose_gonzalez
Databricks Employee
Databricks Employee

hi @Will Blockโ€‹ ,

I think there is a related question being asked in the past. I think it was this one

I found this library, I hope it helps.