Databricks Community

Dimitry

Hi all I have 2 clusters, that look identical but one runs my UDF in parallel another one does not.The ones that do is personal, the bad one is shared.import pandas as pd from datetime import datetime from time import sleep import threading # test f...

Dimitry

I've got an UDF which I call using applyInPandasThat UDF is to distribute API calls.It uses my custom .py library files that make these calls.Everything worked until I use `dbutils.widgets.get` and `dbutils.secrets.get` inside these libraries.It thro...

Dimitry

I've read all relevant articles but none have solution that I could understand. Sorry I'm new to it.I have a simple UDF to demonstrate the problem:df = spark.createDataFrame([(1, 1.0, 'a'), (1, 2.0, 'b'), (2, 3.0, 'c'), (2, 5.0, 'd'), (2, 10.0, 'e')]...

Dimitry

Hi allI'm trialing Databricks for running complex python integration scripts. It will be different data sources (MS SQL, CSV files etc.) that I need to push to a target system via GraphQL. So I selected Databricks vs MS Fabric as it can handle comple...

Dimitry · 05-15-2025

Hi allI'm switching to evaluating Databricks after being unhappy with Microsoft Fabric.I'm looking for a platform to aggregate external files and tables, do processing and push into 3rd party using GraphQL. That push is going to be quite sophisticate...

Dimitry

I ran outbound graphql on serverless, but on Azure version of the Databricks. Azure VMs don't restrict this.My problem with serverless is How to "Python versions in the Spark Connect clien... - Databricks Community - 121213, so serverless is still un...

Dimitry

As a side note "no isolation shared" cluster has no access to unity catalog, so no table queries.I resorted to using personal compute assigned to a group.

Dimitry

I sort of fixed it myself. Screenshot above was incorrect for the shared compute.and the fix was in changing the access mode

Dimitry

Hi mateWhilst I understand what you are saying, I can't see how it can possibly work.I get only 2 choices for the serverless environment version:and version 2 being the latest is python 3.11Serverless environment version 2 - Azure Databricks | Micros...

Databricks Community

User Stats

User Activity

Struggle to parallelize UDF

UDF fails with "No module named 'dbruntime'" when using dbutils

How to "Python versions in the Spark Connect client and server are different. " in UDF

Cannot run merge statement in the notebook

Unable to query external GraphQL API

Re: UDF fails with "No module named 'dbruntime'" when using dbutils

Re: Struggle to parallelize UDF

Re: Struggle to parallelize UDF

Re: How to "Python versions in the Spark Connect client and server are different. " in UDF