Databricks Community

Hi @lingareddy_Alva ,Thank you for your excellent response. I really appreciated it.I couldn't find the mention that says "Instance profiles are still supported but should be used for specific, advanced access cases." I will use it for now, recognizi...

Hi @zmsoft  The event hook provided, user_event_hook, must be a Python callable that accepts exactly one parameter - a dictionary representation of the event that triggered the execution of this event hook. The return value of the event hook has no s...

hi @William_Scardua Here’s a comprehensive overview of how to collect usage and job‐execution metrics in Databricks without Unity Catalog,using REST APIs, audit logs, system tables, and built-in monitoring features.In summary, you can retrieve:1. Job...

Thank you all for your replies. @lingareddy_Alva what about @Walter_C and @mnorland mentioned about enabling serverless tasks. Is this possible and how?

To calculate CPU usage, CPU idle time, and memory usage per cluster per day, you can use the system.compute.node_timeline system table. However, since the data in this table is recorded at per-minute granularity, it’s necessary to aggregate the data ...

Hi @ankit001mittal Yes, you're right, when you enable the "Publish Event Log to Metastore" option for DLT pipelines, Databricks creates a separate event log table for each pipeline. So, if you have thousands of pipelines, you'll see thousands of log ...

Hi holychs,How are you doing today?, As per my understanding, yes, in Databricks Workflows, if you're running a multi-task job (like your parent job triggering multiple child tasks), you can repair only the failed task without restarting the entire j...

@vivi007 Yes, you can create dependencies between jobs in different DABs (Databricks Asset Bundles), but this requires a different approach than task dependencies within a single DAB.Since DABs are designed to be independently deployable units, direc...

use this https://www.youtube.com/watch?v=iqf_QHC7tgQ&list=PL2IsFZBGM_IGpBGqxhkiNyEt4AuJXA0Gg it will help you a lot

The tool is really meant for dbfs and is only accessible from within Databricks.  If I had to guess the idea is that most folks will not be using dbfs for production or sensitive data (for a host of good reasons) and as such there has not been a big ...

@fedemgp I was able to turn the desired setting on and off with Terraform with this code: GitHub Gist I'm using Databricks Terraform provider version 1.74.0 and my Databricks runs on Google Cloud.

Hello @gm_co! Were you able to sort this out?You can display % in two ways: In the General settings, check the box for Normalize values to percentage.As you have enabled Labels, just set the Data labels to {{ @@yPercent }}. This will show the percent...