Hi all,
I hope you could help me to figure out what I am missing.
I'm trying to do a simple thing. To read the data from the data ingestion zone (csv files saved to Azure Storage Account) using the Delta Live Tables pipeline and share the resulting table to another Databricks workspace using Delta Sharing.
Here is the code that describes the DLT pipeline.
import dlt
from pyspark.sql.functions import *
from pyspark.sql.types import *
raw_path = "/mnt/ingestion/sensors-readings"
@dlt.table(
comment = "Contains data received from sensors API"
)
def sensors_raw():
# Auto-loader to load newly ingested files only.
df = spark.readStream.format("cloudFiles") \
.option("cloudFiles.format", "csv") \
.option("header",True) \
.load(raw_path)
return (df)
It runs successfully and the table is added to the target schema.
![vkuznetsov_0-1689259588838.png vkuznetsov_0-1689259588838.png](/t5/image/serverpage/image-id/2847iD3497FE6F2A2ED3B/image-size/medium/is-moderation-mode/true?v=v2&px=400)
But when I try to create a share the table is not displayed.
![2023_07_13_16_48_52_Data_Explorer.png 2023_07_13_16_48_52_Data_Explorer.png](/t5/image/serverpage/image-id/2851iDB2299C07B64755E/image-size/large/is-moderation-mode/true?v=v2&px=999)
The above tables available for sharing are created from the notebook as df.write.saveAsTable().
When reading available documentation I've seen that the STREAMING_TABLE can't be shared via Delta Sharing.
Maybe I'm missing some settings? It would be great if you could help me to figure it out.
Thanks.