Databricks Community

Rishitha · 10-24-2023

I'm trying to addmonotonicallyIncreasingId() column to a streaming table and I see the following errorFailed to start stream [table_name] in either append mode or complete mode. Append mode error: Expression(s): monotonically_increasing_id() is not s...

Rishitha · 09-22-2023

Hi all!I have a question about setting a target schema. How to set different targets for 2 different tables in the same delta live table pipeline. We have 2 target schemas in a database Bronze_chema and silver_schema. The pipeline has a streaming ra...

Rishitha · 05-09-2023

I have data on s3 and i'm using autoloader to load the data. My json docs have fields which are array of structures. When I don't specify any schema the whole data is stored as strings even the array of structures are just a blob of string making it ...

Rishitha · 09-22-2023

@btafur Can we also set the auto_terminate minutes with the policy? (for the dlt cluster type)

Rishitha · 09-22-2023

That's perfect thank you. @btafur

Rishitha · 09-22-2023

Thanks again @btafur Hoping for this feature to release soon!

Rishitha · 09-22-2023

@btafur That helped, the issue is fixed now. Thanks. a follow up question, can we give 2 values there for cluster type, something like this"cluster_type": { "type": "fixed", "value": "dlt" },"cluster_type": { "type": "fixed", "valu...

Rishitha · 09-22-2023

I have also asked the question in the office hours, they said it will be a new feature, announced in the summit too, but there is a workaround to do it now. Anyone implemented the workaround? '

Databricks Community

User Stats

User Activity

Delta live tables straming

DLT pipeline

Normalizing data from autoloader

Re: Cluster policy not showing while creating delta live table pipeline

Re: Cluster policy not showing while creating delta live table pipeline

Re: DLT pipeline

Re: Cluster policy not showing while creating delta live table pipeline

Re: DLT pipeline