Databricks Community

sachamourier · 07-28-2025

Hello,I am trying to install and use the library GraphFrames but keep receiving the following error: "AttributeError: 'SparkSession' object has no attribute '_sc'".I have tried to install the library on my all-purpose cluster (Access mode: Standard)....

sachamourier · 07-03-2025

Hello,For one of my clients, we are using an all-purpose cluster to run some Databricks notebooks. We noticed in the logs of the cluster some Azure Quota Exceptions, from which we would like to know more.As you can see attached, the cluster always su...

sachamourier · 06-30-2025

Hello all,We are currently running Azure Databricks notebooks through Azure Data Factory pipelines, where we use job clusters defined through policies. As an admin, I am able to see the list of all job clusters that have created when ADF makes a call...

sachamourier · 02-25-2025

Hello,I am trying to install some wheel file and requirements.txt file from my Unity Catalog Volumes on my Databricks job cluster using an init script but the results are very inconsistent.Does anyone have ever faced that ?What's wrong with my approa...

sachamourier · 02-12-2025

Hello,I would like to install some libraries (both public and private) on a job cluster. I am using Azure Data Factory to run my Databricks notebooks and hence would like to use job clusters to run these jobs.I have passed my init script to the job c...

sachamourier · 07-29-2025

@szymon_dybczak Thanks for the responses. I indeed changed my all-purpose cluster access mode and it worked. I figured that was a nicest option than changing the runtime.

sachamourier · 07-04-2025

@szymon_dybczak That makes sense thank you ! However, do you have any idea of why it shows "Current usage: 0" ? And also why does the cluster still manage to reach the max number of nodes even with these "failures" ? Sacha

sachamourier · 07-04-2025

Hi @szymon_dybczak ,Thank you for your help. I have followed your suggested steps and have the following output:Should this increased then ? It seems like it's not being used though...Sacha

sachamourier · 07-04-2025

Hi @Vidhi_Khaitan ,Thank you for your response.Attached is the type of cluster we are using. What quotas should we increase within the Azure Portal for this type of cluster family ? Also, within the Azure Portal Quotas page, no usage is shown for our...

sachamourier · 06-30-2025

Hello @Isi ,Thank you for your help ! I noticed the possibility to do that on the right-hand sidebar, but this is going to only provide access to a previously run job with a given ID, and not the future ones I believe.For example, attached, you can s...

Databricks Community

User Stats

User Activity

Enable to use library GraphFrames

All-purpose cluster upsize failures

Job clusters view permissions

Install Python libraries on Databricks job cluster

Use init script for Databricks job cluster via Azure Data Factory

Re: Enable to use library GraphFrames

Re: All-purpose cluster upsize failures

Re: All-purpose cluster upsize failures

Re: All-purpose cluster upsize failures

Re: Job clusters view permissions