Databricks Community

loic · 02-06-2025

Hello,I am facing an issue that I do not understand. I have as simple Scala notebook with a "read function" that reads a json file on an external storage and does few changes to this DataFrame. I do my test on "all purpose" compute, DS3v2 (14gig/4cor...

loic · 01-29-2025

Hello,We are forwarding this Microsoft tutorial to secure our storage access:https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-security/serverless-firewallWe have a weird behavior when we create several NCCs in th...

loic · 03-28-2025

@Ria1 I am not an expert but this is my understanding.When you speak of "Vnet", you speak of "vnet injection" right? So here, it is only the private and public subnets of your "compute plan" that are injected in your VNET.This update is only about ou...

loic · 02-14-2025

Hello @shiv_DB25 According to me, your NSG config is fine. The rule 65001 on your outbound should allow your cluster to access internet.Not sure it will help, but there is this line in your log:'Error code: ERROR_NO_MATCHING_DISTRIBUTION'I don't know...

loic · 02-07-2025

Finally, we understood the issue by ourself.By default, Databricks create new session for each new job. It is possible to change this behavior with the spark configuration (to put in spark config section of the compute settings):spark.databricks.sess...

loic · 02-06-2025

Ok, so no, I correctly set the subnets of my NCC in the Virtual Networks setting as documented:https://learn.microsoft.com/en-us/azure/databricks/security/network/serverless-network-security/serverless-firewallThis setting is working fine, without th...

loic · 02-04-2025

When you say: "I have a Databricks pyspark query that has been running fine for the last two weeks but am now getting the following error despite no changes to the query: OutOfMemoryError: GC overhead limit exceeded."Can you tell us how do you execut...

Databricks Community

User Stats

User Activity

Several executions of a single notebook lead to java.lang.OutOfMemoryError

Different NCC having same subnets

Re: [Azure] Outbound public IP address ranges for the control plane will be updated May 20, 2025

Re: We are opening additional ports for Azure Databricks

Re: Several executions of a single notebook lead to java.lang.OutOfMemoryError

Re: Different NCC having same subnets

Re: Databricks PySpark error: OutOfMemoryError: GC overhead limit exceeded