SQLSTATE: XX000 The Spark SQL phase planning failed with an internal error.

ÓscarHernández
New Contributor II

‎04-22-2025 07:05 AM

Hello everyone,

I am currently working with a SQL Warehouse and have been getting the following error message:
[INTERNAL_ERROR ] The Spark SQL phase planning failed with an internal error. You hit a bug in Spark or the Spark plugins you use. Please, report this bug to the corresponding communities or vendors, and provide the full stack trace. SQLSTATE: XX000

I get said message when using a SQL table function created by myself in conjunction with a table. I am using 2 different queries trying to get any results, but both queries return the same error. The queries used are:

 
SELECT *
FROM mytable A
LEFT JOIN lateral myfunction(A.column1, A.column2) B ON TRUE

SELECT *
FROM mytable A
CROSS JOIN LATERAL (
    SELECT *
    FROM myfunction(A.column1, A.column2)
) AS B;
 
Does anyone know how to use a tabular table whose parameters are passed from a table's column?
 
PS: the tabular function works perfectly when used directly select * from myfunction(1, 2)
 
Thanks in advance!
0 Kudos

ÓscarHernández
New Contributor II

‎04-23-2025 10:54 PM

I have tried to simplify the query as much as possible to see if that helps but the bug still persists. The problem should be something with the way Databricks treats columns passed as arguments for a function.

I tried these queries:

select * FROM VALUES (183915595, 231504) AS A (column1, column2)
LEFT JOIN lateral myfunction(A.column1, A.column2) FIP ON TRUE;
 
SELECT B.*, A.*
FROM VALUES (183915595, 231504) AS A (column1, column2)
CROSS JOIN LATERAL (
    SELECT *
    FROM myfunction(A.column1, A.column2)
) AS B;
 
Has anyone else had problems when using tabular functions with columns passed as arguments?
Thanks in advance! 
0 Kudos

Renu_
Valued Contributor II
In response to ÓscarHernández

‎04-24-2025 03:13 AM

Hi @ÓscarHernández, have you tested this issue across different DBR versions, and do you observe the same behavior?

0 Kudos

ÓscarHernández
New Contributor II
In response to Renu_

‎04-24-2025 06:35 AM

Hi @Renu_ , I am using a SQL Warehouse where there is no possibility of choosing an specific DBR (at least I don't know how to do it).

0 Kudos