topic Re: SQLSTATE: XX000 The Spark SQL phase planning failed with an internal error. in Data Engineering

SQLSTATE: XX000 The Spark SQL phase planning failed with an internal error.

ÓscarHernández — Tue, 22 Apr 2025 14:05:58 GMT

Hello everyone,

I am currently working with a SQL Warehouse and have been getting the following error message:
[INTERNAL_ERROR ] The Spark SQL phase planning failed with an internal error. You hit a bug in Spark or the Spark plugins you use. Please, report this bug to the corresponding communities or vendors, and provide the full stack trace. SQLSTATE: XX000

I get said message when using a SQL table function created by myself in conjunction with a table. I am using 2 different queries trying to get any results, but both queries return the same error. The queries used are:

SELECT *

FROM mytable A

LEFT JOIN lateral myfunction(A.column1, A.column2) B ON TRUE

SELECT *

FROM mytable A

CROSS JOIN LATERAL (

SELECT *

FROM myfunction(A.column1, A.column2)

) AS B;

Does anyone know how to use a tabular table whose parameters are passed from a table's column?

PS: the tabular function works perfectly when used directly select * from myfunction(1, 2)

Thanks in advance!

Re: SQLSTATE: XX000 The Spark SQL phase planning failed with an internal error.

ÓscarHernández — Thu, 24 Apr 2025 05:54:16 GMT

I have tried to simplify the query as much as possible to see if that helps but the bug still persists. The problem should be something with the way Databricks treats columns passed as arguments for a function.

I tried these queries:

select * FROM VALUES (183915595, 231504) AS A (column1, column2)

LEFT JOIN lateral myfunction(A.column1, A.column2) FIP ON TRUE;

SELECT B.*, A.*

FROM VALUES (183915595, 231504) AS A (column1, column2)

CROSS JOIN LATERAL (

SELECT *

FROM myfunction(A.column1, A.column2)

) AS B;

Has anyone else had problems when using tabular functions with columns passed as arguments?

Thanks in advance!

Re: SQLSTATE: XX000 The Spark SQL phase planning failed with an internal error.

Renu_ — Thu, 24 Apr 2025 10:13:06 GMT

Hi @ÓscarHernández, have you tested this issue across different DBR versions, and do you observe the same behavior?

Re: SQLSTATE: XX000 The Spark SQL phase planning failed with an internal error.

ÓscarHernández — Thu, 24 Apr 2025 13:35:20 GMT

Hi @Renu_ , I am using a SQL Warehouse where there is no possibility of choosing an specific DBR (at least I don't know how to do it).