cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Problem sharing a streaming table created in Delta Live Table via Delta Sharing

vkuznetsov
New Contributor III

Hi all,

I hope you could help me to figure out what I am missing.

I'm trying to do a simple thing. To read the data from the data ingestion zone (csv files saved to Azure Storage Account) using the Delta Live Tables pipeline and share the resulting table to another Databricks workspace using Delta Sharing.

Here is the code that describes the DLT pipeline.

import dlt
from pyspark.sql.functions import *
from pyspark.sql.types import *

raw_path = "/mnt/ingestion/sensors-readings"

@dlt.table(
    comment = "Contains data received from sensors API"
)
def sensors_raw():
    # Auto-loader to load newly ingested files only.
    df = spark.readStream.format("cloudFiles") \
            .option("cloudFiles.format", "csv") \
            .option("header",True) \
            .load(raw_path)
    return (df)

It runs successfully and the table is added to the target schema.

vkuznetsov_0-1689259588838.png

But when I try to create a share the table is not displayed.

 

2023_07_13_16_48_52_Data_Explorer.png

The above tables available for sharing are created from the notebook as df.write.saveAsTable().

When reading available documentation I've seen that the STREAMING_TABLE can't be shared via Delta Sharing.

Maybe I'm missing some settings? It would be great if you could help me to figure it out.

Thanks.

1 REPLY 1

vkuznetsov
New Contributor III

Sorry, I think I've created the post in the wrong thread. Created the same post in the Community Cove.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!