Databricks Community

Define "parameters" in job as usual and then, try to capture them in DLT by using similar code to this one:dlt.conf.get("PARAMETER_NAME", "PARAMETER_DEFAULT_VALUE")It should get parameter values from job if value exists, otherwise it'll set the defau...

Clearly, you don't have "admin account" permissions. Try to click in the workspace drop-down and then, check if you can see and click in "Manage Account" to confirm BUT it will be very likely you are not allowed to access.You must be Azure Global Adm...

So, @leenack best option so far is to refactor part of your code from stored procedures to functions, specifically the part of querying data. Exactly the same, I proposed in previous comments. Thanks @matt for your response.

Hey!!!Cold starts on Azure Functions Premium can still bite if your instances go idle long enough — even with pre-warmed instances.What usually helps is bumping the `preWarmedInstanceCount` to at least 1 per plan (so there’s always a warm worker), an...

You’ve hit two limitations:Streaming tables don’t allow SET OWNER – ownership cannot be changed.Lakeflow pipeline ID changes require pipeline-level permissions – if you’re not the pipeline owner, you can’t run ALTER STREAMING TABLE ... SET PIPELINE_I...

Hey! Great question — I’ve run into this issue quite a few times while working with large datasets in Databricks, and out-of-memory errors can be a real headache. One of the biggest things that helps is making sure your cluster configuration matches ...

Databricks has recently changed how unresolved column references are handled in PySpark on All-purpose compute clusters. In earlier Databricks Runtime (DBR) 14.3 LTS builds, referencing a non-existent column—such as:   python df = spark.tabl...

I am using the Python SDK and to authenticate I am using a SP and a Secret. Here is the code that worked for me in Azure Databricks notebook.from databricks.sdk import AccountClient acct_client = AccountClient( host="https://accounts.azuredatabr...

You are encountering an error because the default SQL Server JDBC driver bundled with Databricks may not fully support the authentication value "ActiveDirectoryManagedIdentity"—this option requires at least version 10.2.0 of the Microsoft SQL Server ...

The intermittent Internal 500 errors you’re experiencing when accessing the vector search endpoint through a Streamlit app on Databricks—while direct console queries work—suggest an issue with the interaction between your Streamlit app’s environment ...

The error you are seeing means Databricks cannot capture DDL (table definition) changes, even though CDC (Change Data Capture) and CT (Change Tracking) are enabled. You must run the specific DDL support objects script for Databricks ingestion and the...