Databricks Community

serelk · 11-09-2023

I’m keen to learn more about employing Distributed Ledger Technology (DLT) in a production environment. Databricks is indeed promoting Delta Live Tables (DLT) as a comprehensive framework for creating robust, maintainable, and testable data processin...

serelk · 11-07-2023

Hello Databricks Community,I'm working with a DLT pipeline where I consume Protobuf-serialized messages and attempt to decode them using the Spark "from_protobuf" function. My function kafka_msg_dlt_view is outlined as follows:def kafka_msg_dlt_view(...

serelk · 11-09-2023

Just a small remark: I definitely meant Delta Live Tables, not Distributed Ledger Technology (DLT)

serelk · 11-09-2023

I finally found the appropriate method for configuring the PERMISSIVE mode. With this setup, corrupted protobuf messages will be processed without throwing an exception .withColumn("msg",from_protobuf(col("value"), message_name, desc_file_path, {"mo...

serelk · 09-12-2023

Same here

Databricks Community

User Stats

User Activity

Delta Live Tables (DLT) in a production environment

Handling Schema Mismatch in DLT Pipeline with from_protobuf Function

Re: Delta Live Tables (DLT) in a production environment

Re: Handling Schema Mismatch in DLT Pipeline with from_protobuf Function

Re: Run Delta Live Tables as service principal