Databricks Community

Hey @amitpm According to the documentation, this feature is currently in Public Preview, so if your Databricks account has access to public preview features, you can reach out to support to enable it and start testing performance.Setup guide for Lake...

Hi,In my opinion, Databricks Deep Clone does not currently support cloning Unity Catalog tables natively across different metastores (each region having its own metastore). Deep Clone requires that both source and target belong to the same metastore ...

@arun_6482 The error you have shared suggests that there is a network issue in your Databricks deployment within your AWS account. Please review the documentation provided below and ensure that all your routes and ports are configured correctly. Doc:...

Hey @kavithai Sometimes there are limitations in the laws of each country regarding "sharing" data outside private clouds or regions, which make it impossible to transmit data outside of your private networks. This is especially true for banks, which...

hello @noorbasha534 Unfortunately, in SQL Warehouses, you can't attach an init script that automatically runs when the warehouse starts (similar to what you can do with clusters).However, there are a few alternatives you can consider:Session Variable...

I would say if the data is not complex and you are not handling any DQ checks in the pipeline the go for a regular databricks workflow and save it as delta table since you are refreshing the data every 1 month and it is not streaming workload.

A couple of things to explore here as it can be solved a couple of different ways.1. A workspace admin needs to update your Entitlements to allow for cluster creation.  This is generally not a best practice as it can lead to unmanaged cluster sprawl....

Hi @Monteiro_12 ,As far as I know, DLT pipeline doesn’t support adding a certified tag directly through table properties or pipeline configurations. Tags like system.Certified needs to be applied manually after the table is created via SQL

The issue is happening because you're calling logging.getLogger(__name__) before setting up logging.basicConfig(). When the logger is created too early, it doesn't know about the file handler, so it doesn't write to the file.To fix this, make sure yo...

Use Azure SQL Spark Connector. This method allows Databricks to read from and write to Azure SQL Database efficiently, supporting both bulk operations and secure authentication.Azure sql : Install connector, configure JDBC, use Key Vault, set permiss...