cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Databricks SQL Exists does not work correct?

Sergecom
New Contributor II

Can someone explain to me how this is possible?

Sergecom_0-1742460269477.png

Sergecom_1-1742460306964.png

 

 

7 REPLIES 7

TheShua
Databricks Employee
Databricks Employee

I replicated the query exactly but I couldn't produce the same results, and got back the expected result of the single row with 'aaa'. My initial thought is that it could be a bug in the runtime. Can you share the runtime of the cluster you are using?

Sergecom
New Contributor II

Thanks for your reply @TheShua !
I use 15.4 LTS
Here is the summary of cluster configuration:

Sergecom_0-1742506166174.png

 

TheShua
Databricks Employee
Databricks Employee

@Sergecom This looks to be a possible bug, thank you for flagging. It looks like the EXISTS function works as intended with Serverless Compute and Serverless SQL Warehouses, but exhibits a different behavior when run on all-purpose compute. I've raised a ticket so that they can look into this.

Sergecom
New Contributor II

Thanks for the update!  Let me know if you need any more details from my side—happy to help.
Is there any way to follow this ticket @TheShua ?

marco1995
Visitor

We were able to reproduce the problem, but it only occurs in Databricks SQL version 2025.10. With version 2024.05, the correct result is obtained.

Sergecom
New Contributor II

Thanks for feedback @MarcoI've also tested with an older All-purpose Runtime 12.2 LTS and it works.
But its alarming that the latest runtime does not work as expected.

I will post SQL code so it will be easier for other to test. Both these queries give wrong results:

%sql
WITH a AS (
    SELECT 'aaa' as produktkod
    UNION ALL
    SELECT 'bbb'  as produktkod
),
b AS (
    SELECT 'aaa' as produktkod
    UNION ALL
    SELECT 'ccc'  as produktkod
)
SELECT a.*
FROM a
WHERE exists (
  select 1 from b where a.produktkod = b.produktkod
);

%sql
WITH a AS (
SELECT 'aaa' as produktkod
UNION ALL
SELECT 'bbb' as produktkod
),
b AS (
SELECT 'aaa' as produktkod
UNION ALL
SELECT 'ccc' as produktkod
)
SELECT a.*,
(SELECT COUNT(1)
FROM b
WHERE b.produktkod = a.produktkod) as match_count
FROM a;
 

TheShua
Databricks Employee
Databricks Employee

@Sergecom The engineering team worked on this and I believe it is now resolved for the newer runtimes. I confirmed it nows works as expected on 16.2. Can you confirm if it works on your end?

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group