Databricks Community

Chengcheng · 06-29-2023

Hi everone, I'm trying to query data in Azure Synapse Dedicated SQL Pool according to the documentaion using:.format("com.databricks.spark.sqldw") Query data in Azure Synapse AnalyticsIt says that a abfss temporary location is needed.But I found that...

Chengcheng · 06-16-2023

I am building a data pipeline using Delta Live table in Azure Databricks to move data from a raw data table to a feature table and model inference results table. However, I am concerned about the potential for duplication issues in future operations,...

Chengcheng · 06-15-2023

Hi, I tried to deploy a Feature Store packaged model into Delta Live Table using mlflow.pyfunc.spark_udf in Azure Databricks. This model is built by Databricks autoML with joined Feature Table inside it.And I'm trying to make prediction using the fol...

Chengcheng · 07-05-2023

Sorry to hear that. I thought it should work. I don't have any other idea.

Chengcheng · 07-03-2023

Hi AbhiJ, You can use Select tables for refreshin your Delta Live Table UI, it allows you to select tables you would like to refresh. And "Refresh Table <TableName>" may not work like you are thinking it is.It's not used to refresh Delta Live Table,...

Chengcheng · 07-03-2023

You can try to remove you from the users in Admin Settings. But since you can't remove yourself. You will need someone else who is workspace admin to do this.

Chengcheng · 07-03-2023

If you want to update DLT asynchronously.(Without any manual intervention)The only way is to separate it into two pipeline and schedule these two DLT pipeline separately.Since DLT don't support any conditional run for now. You can't decide not to fre...

Chengcheng · 06-16-2023

If your source table is a delta table or cloud file where DLT can monitor any changes. The answer would YES I think. That is exactly what DLT Materialized view does.See documentation of DLT for details:https://learn.microsoft.com/en-us/azure/databric...

Databricks Community

User Stats

User Activity

The default location of temporary file in Azure Synapse Connector(com.databricks.spark.sqldw)

Best practice of handling Data Duplication in a Delta Live Table Pipeline with daily batch data source.

Is Feature Store packaged model compatible with Spark UDF?

Re: How to hide workspace to which I don't have access anymore?

Re: How to refresh materialized view (Delta Live Table) without running a pipeline every time.

Re: How to hide workspace to which I don't have access anymore?

Re: How to refresh materialized view (Delta Live Table) without running a pipeline every time.

Re: Using DLT with a non-streaming large table