Databricks Community

Hi,Current approach reloads dim_df in every batch, which can be inefficient. To optimize, consider broadcasting dim_df if it's small or using a mapGroupsWithState function for stateful joins. Also, ensure that fact_df has sufficient watermarking to h...

import json def get_job_context(): """Retrieve job-related context from the current Databricks notebook.""" # Retrieve the notebook context ctx = dbutils.notebook.entry_point.getDbutils().notebook().getContext() # Convert the context...

Hi @manoj_2355ca , I am also facing the same error, did you get the solution for it?

@SaadhikaB Hi, when I run  dbutils.library.restartPython(), I get the following error 

"Deploying a custom serving endpoint for LLMs can be challenging, especially when handling model dependencies and scaling issues. Has anyone found a reliable workaround for deployment failures? Also, for those looking for updates on government assist...

Hello @nikhil_2212! It looks like this post duplicates the one you recently posted. A response has already been provided to the Original post. I recommend continuing the discussion in that thread to keep the conversation focused and organised.

We have customers that read millions of files per hour+ using Databricks Auto Loader. For high-volume use cases, we recommend enabling file notification mode, which, instead of continuously performing list operations on the filesystem, uses cloud nat...

Hello @Pooviond! Please submit a ticket with the Databricks Support team for assistance in resolving this issue.