- 1174 Views
- 2 replies
- 0 kudos
Azure databricks API and DLT databricks
how can i pass parameter from Azure data factory rest web API to delta live Databricks?I get this error: "Py4JJavaError: An error occurred while calling o382.getArgument.: com.databricks.dbutils_v1.InputWidgetNotDefined: No input widget named *** def...
- 1174 Views
- 2 replies
- 0 kudos
- 0 kudos
@girl_chan The error you are encountering usually occurs when a widget referenced in a Databricks notebook is not defined. In your case, the issue is likely related to how you pass parameters from Azure Data Factory (ADF) to Delta Live Tables (DLT) i...
- 0 kudos
- 8783 Views
- 5 replies
- 5 kudos
DLT with unity catalog and ML
We are currently using DLT with unity catalog. DLT tables are created as materialized views in a schema inside a catalog. When we try to access these materialized view using a ML runtime (ex. 13.0 ML) cluster, it says, that we must use Single User se...
- 8783 Views
- 5 replies
- 5 kudos
- 5 kudos
No updates as far as I am aware.You could make the workflow copying the data smart though and try to only do incremental updates, seems like a lot of effort though.
- 5 kudos
- 6353 Views
- 2 replies
- 1 kudos
Isolation Forest prediction failing DLT pipeline, the same model works fine when prediction is done outside DLT pipeline.
Hey community membersI am new to Databricks and was building a simple DLT pipleine that loads data from S3 and runs an Isolation forest prediction to detect anomalies. The model has been stored in Model Registry. Here's the code for the pipeline:@dlt...
- 6353 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Mukul Degweker​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
- 1 kudos
- 2029 Views
- 5 replies
- 1 kudos
Latest Blog PostsJanuary 13 - 20Â Did you get a chance to look at the most recent blog posts? Here are some happening content from the past week that i...
Latest Blog PostsJanuary 13 - 20Did you get a chance to look at the most recent blog posts? Here are some happening content from the past week that is worth the read. What’s New With SQL User-Defined Functions In this blog, we describe several enhanc...
- 2029 Views
- 5 replies
- 1 kudos
- 1 kudos
Thanks @Sujitha Ramamoorthy​ , for sharing with the community these are worth reading and insightful.
- 1 kudos
- 1419 Views
- 1 replies
- 4 kudos
Stream data from Delta tables replicated with Fivetran into DLT
I'm attempting to stream into a DLT pipeline with data replicated from Fivetran directly into Delta tables in another database than the one that the DLT pipeline uses. This is not an aggregate, and I don't want to recompute the entire data model eac...
- 1419 Views
- 1 replies
- 4 kudos
- 4 kudos
Hi @M A​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question first. Or else bricksters will get back to you soon. Thanks
- 4 kudos
- 15843 Views
- 1 replies
- 5 kudos
Resolved! ingest a .csv file with spaces in column names using Delta Live into a streaming table
How do I ingest a .csv file with spaces in column names using Delta Live into a streaming table? All of the fields should be read using the default behavior .csv files for DLT autoloader - as strings. Running the pipeline gives me an error about in...
- 15843 Views
- 1 replies
- 5 kudos
- 5 kudos
After additional googling on "withColumnRenamed", I was able to replace all spaces in column names with "_" all at once by using select and alias instead:@dlt.view( comment="" ) def vw_raw(): return ( spark.readStream.format("cloudF...
- 5 kudos