Databricks Community

>  I need to retrieve a result of the the notebook taskIf you want to know if the task run has succeeded or not, you can enable the "lakeflow" system schema and you'll find the logs of jobs and task runs.You could then use the above info to execute a...

you can use forEachBatch to write an arbitrary table. See this page for an example of running arbitrary SQL code during an autoloader/streaming run.

Same problem here.

You are correct, the intention of this port is to connect to the Hive metastore 

Databricks allocates resources to executors on a node based on several factors, and it appears that your cluster configuration is using default settings since no specific Spark configurations were provided. Executor Memory Allocation: The spark.exec...

I get an invalid credentials error on using this method. I have generated a token in databricks, any idea why?

i think the dab mlop stack template is pretty helpful re: how to bundle, schedule and trigger custom jobshttps://docs.databricks.com/en/dev-tools/bundles/mlops-stacks.htmlyou can bundle init it locally and it should give you the skeleton of how to bu...

The code follows similar pattern below to load the different tables. import dltimport reimport pyspark.sql.functions as Flanding_zone = '/Volumes/bronze_dev/landing_zone/'source = 'addresses'@Dlt.table(comment="addresses snapshot",name="addresses")de...

Hi @tonyd Thank you for reaching out to the Databricks Community. You are trying to create a Serverless Generic Compute Cluster which is not supported. You cannot create a Serverless compute Cluster. As per the below link, if you observe, there is no...

Ah, I have had that same error before when cloning from git.  I'm guessing you got the Repo URL by hitting the "clone" button in ADO and copy/pasting it in to Databricks.  One thing I've done in the past is in the Repository URL, remove the "org@" pa...