- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-11-2025 01:07 PM
I tried the following and got an error. What's the issue?
Here is the registered function the agent will call:
def myFunction(col_val1: str, col_val2: str) -> str:
"""
A function that will accept 2 strings from caller and insert into a table.
"""
from pyspark.sql.session import SparkSession
spark = SparkSession.builder.getOrCreate()
spark.sql(f"CALL myCatalog.mySchema.myStoredProcedure('{col_val1}', '{col_val2}');")
return "Data inserted successfully."
Then from Playground, load the Tool to the above function and ask agent to insert data. Got the below error:
== Error ==
SystemExit: -1
== Stacktrace ==
File "<udfbody>", line 3, in main
spark = SparkSession.builder.getOrCreate()
File "/databricks/spark/python/pyspark/sql/session.py", line 574, in getOrCreate
else SparkContext.getOrCreate(sparkConf)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/databricks/spark/python/pyspark/core/context.py", line 579, in getOrCreate
SparkContext(conf=conf or SparkConf())
File "/databricks/spark/python/pyspark/core/context.py", line 207, in __init__
SparkContext._ensure_initialized(self, gateway=gateway, conf=conf)
File "/databricks/spark/python/pyspark/core/context.py", line 500, in _ensure_initialized
SparkContext._gateway = gateway or launch_gateway(conf)
^^^^^^^^^^^^^^^^^^^^
File "/databricks/spark/python/pyspark/java_gateway.py", line 63, in launch_gateway
SPARK_HOME = _find_spark_home()
^^^^^^^^^^^^^^^^^^
File "/databricks/spark/python/pyspark/find_spark_home.py", line 91, in _find_spark_home
sys.exit(-1) SQLSTATE: 39000
== SQL (line 1, position 😎 ==
SELECT `myCatalog`.`mySchema`.`myStoredProcedure`("test1","test2") as output
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Error: Error: org.apache.spark.SparkRuntimeException: [UDF_USER_CODE_ERROR.GENERIC] Execution of function myCatalog.mySchema.myFunction(test1, test2) failed.