Databricks Community

Hi @397973, Combining multiple notebooks into a single notebook isn't an out-of-the-box feature, but will try to combine %run commands ando output them to see if it works, sort of like: %run "/path/to/notebook1"%run "/path/to/notebook2"

When you create a Databricks Lakehouse Monitoring monitor with an Inference profile, the system automatically generates two metric tables: a profile metrics table and a drift metrics table. Here's how this process works: Background Processing When yo...

@ShivangiB You're correct in your understanding. When you change a clustering key using ALTER TABLE followed by OPTIMIZE, it doesn't automatically recluster existing data. Let me explain why this happens and what options you have.In Delta Lake (which...

Hello @HarryRichard08! It looks like this post duplicates the one you recently posted. A response has already been provided to the Original post. I recommend continuing the discussion in that thread to keep the conversation focused and organized.

It turns out, the problem is the documentation. It says that the permission parameter (that's supplied in) is an array of strings. It really just expects a string, either UNKNOWN, USER, or ADMIN. It would be great if the team could fix the documentat...

Hi @gauravmahajan,Most of the cost / DBU used can be retrieved from System tables across your different workspaces in a databricks account. Details related to job compute types and it's associated cost can be fetched from the queries mentioned in the...

@SB93 The error message you are seeing indicates that the cluster failed to launch because the Spark driver was unresponsive, with possible causes being library conflicts, incorrect metastore configuration, or other configuration issues. Given that t...

@Suncat There hasn't been any major changes for than a year: https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-version-support E.g. I don't believe we will see support for Spark 3.5 at all. At least, apparently it's support...

Hi @Rachana2,As @Alberto_Umana has mentioned I'd check table_lineage / column_lineage tables, as maintaining a lineage through a bespoke pipeline/tooling may not be a right approach.Can you please explain your use case which explains why you don't wa...