Assistance Needed: Issues with Databricks SQL Queries and Performance

[boitumelodikoko]

Hi everyone,

I hope you're all doing well.

I'm experiencing some challenges with Databricks SQL, and I wanted to reach out to see if others have encountered similar issues or have suggestions for troubleshooting. Below is a summary of the problems I'm facing:

Issues Observed

1. Error Messages and Query Loading:

• Error tabs frequently appear while working on existing queries (see attached screenshots).
• Queries fail to load with the message: "Unable to load query. Please check you have permissions and that the query exists." Reloading does not resolve the issue.

2. Editor Switching:

• The platform randomly switches between the classic SQL editor and the new one, especially when working with a mix of saved queries from the classic editor and newly created ones.

3. Performance Problems:

• Tabs sometimes close unexpectedly or fail to display correctly.
• Query execution times are inconsistent, with some queries taking significantly longer than usual to run.

Steps Taken

I've already tried clearing my browser cache and cookies (using Chrome), but the issues persist.

Request for Assistance

I've attached screenshots to provide more context on the behaviour. Has anyone experienced similar issues, or is this a known problem? Any advice or guidance on next steps would be greatly appreciated.

Thank you in advance for your support!

Best regards,
Boitumelo

[Walter_C]

After hitting on Reload the query becomes available but you see that the SQL editor changes to classic or to the new editor as well.

I have noticed some of this behavior when the query is not supported on any of either views so the system will try to jump in between each other to make it work.

[boitumelodikoko]

@Walter_C, thanks for the response. Do you perhaps know if there is a potential remedy for this behaviour? 

I also wonder if the Databricks team is working on a fix.

[Walter_C]

I will check on this with our internal team.

[boitumelodikoko]

Hi @Walter_C,

Any updates?

[boitumelodikoko]

Thanks @Walter_C, looking forward to hearing back from you.

[Walter_C]

Hello, yes, they have informed to us that this is known and they are working to integrate all queries in the new format, there is still no ETA but they are working on it

[boitumelodikoko]

Hi @Walter_C,

Thanks for the feedback. Could you perhaps find out what the issue is exactly and what the current workaround is?

This will help with giving direction to the rest of my team.

Regards,

Boitumelo

[Walter_C · Accepted solution]

This is expected behavior for now while we are still improving the new SQL editor. Eventually, the new SQL editor will be the only editor. An individual user can toggle apply to all my queries which should let them work primarily in the new SQL editor, but if they are working on a shared query they may occasionally still need to go back to the old editor

[boitumelodikoko · Accepted solution]

Hi @Walter_C,

Thank you for your input and support regarding the challenges I’ve been experiencing with Databricks SQL.

I followed up with support, and they confirmed that these are known issues currently under review. Here’s a summary of the response:

1. Known Issue with Clusters and Data Versions:

   • Some clusters might operate on older versions of the data, leading to discrepancies in query results.

2. New SQL Editor Limitations:

   • The new SQL editor cannot be used for all queries. Specific limitations include:
     • Queries not owned by the user.
     • Queries containing Query-based Dropdown Lists or Range parameter types.
   • Detailed guidance and workarounds can be found in the SQL Editor release notes and the New SQL Editor documentation.

3. Current Workarounds:

   • Enable the new editor only for queries you own.
   • Adjust queries to avoid unsupported parameter types if you wish to use the new editor.

The support team is actively working to integrate all queries into the new editor, though no ETA has been provided. If anyone encounters similar issues, I recommend reviewing the documents shared by Microsoft for more clarity.

If you have any additional insights or workarounds, please feel free to share!

Thanks again for the assistance.