- 2421 Views
- 6 replies
- 0 kudos
Resolved! Spark connect client and server versions should be same for executing UDFs
I am trying to execute pandas UDF in databricks. It gives me the following error on serverless compute,File /local_disk0/.ephemeral_nfs/envs/pythonEnv-b11ff17c-9b25-4ccb-927d-06a7d1ca7221/lib/python3.11/site-packages/pyspark/sql/connect/client/core.p...
- 2421 Views
- 6 replies
- 0 kudos
- 0 kudos
@chinmay0924 - You can change the serverless client image by selecting the environment panel in a Databricks notebook (on the right pane) or in the "Environment and Libraries" section while configuring a Databricks Job Task. The set of available serv...
- 0 kudos
- 2475 Views
- 2 replies
- 1 kudos
Private Python Package in Serverless Job
I am trying to create a Databricks Job using Serverless Compute. I am using wheel file to run the Python Job.The wheel file has setup.py file using which all dependencies are installed. One of the package dependency is a private package hosted on Git...
- 2475 Views
- 2 replies
- 1 kudos
- 3350 Views
- 0 replies
- 1 kudos
Webinar: Goodbye, Data Warehouse. Hello, Lakehouse. The data warehouse had a good run. But now there’s a simpler, cheaper and more efficient alternati...
Webinar: Goodbye, Data Warehouse. Hello, Lakehouse.The data warehouse had a good run. But now there’s a simpler, cheaper and more efficient alternative: the lakehouse. Come and learn all about it in a webinar on May 18, 2023.If you’re still running s...
- 3350 Views
- 0 replies
- 1 kudos