โ12-20-2024 03:05 AM - edited โ12-20-2024 03:06 AM
Hi Community,
Iโve encountered an issue with the Databricks Extension for VS Code that seems to contradict the documentation. According to the Databricks documentation, the extension supports magic commands like %sql when used with Databricks Connect:
However, in practice, this functionality doesnโt appear to work as expected.
Hereโs an example of the code I ran:
# Databricks notebook source
df = spark.sql("SELECT 1 AS from__python_cell")
df.show()
# COMMAND ----------
# MAGIC %sql
# MAGIC SELECT 1 AS from_sql_cell
The first Python cell executes correctly, displaying the expected result. However, the %sql magic command produces an error:
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
E0000 00:00:1734690770.134581 5908 init.cc:229] grpc_wait_for_shutdown_with_timeout() timed out.
This raises several questions:
My setup includes:
โ12-28-2024 08:50 AM
are you trying to see it from the interactive screen as following screenshot:
โ12-20-2024 04:03 AM
According to the documentation, the Databricks extension for Visual Studio Code supports the %sql magic command.
However, there are limitations to this support:
Given your setup (Databricks Connect: 13.3.2, Databricks Runtime: 16.0, Databricks Extension for VS Code: v2.5.0), it appears that the %sql magic command should be supported.
Do you have option to open support case so this can be reviewed further and possibly engage engineering if a bug is detected?
โ12-20-2024 04:17 AM
Hi @Walter_C ,
Thanks for the answer!
Just to update you I have also adjusted the versions so that all versions match:
The extensions shows everything as green:
The error is still the same:
I am not able to submit the case or at least I am not sure how to do it considering I am using my private Azure Databricks and private e-mail.
Help centre message:
โ12-20-2024 05:58 AM
Got it, I will check with my internal team to validate if there is any issue around.
โ12-20-2024 06:34 AM - edited โ12-20-2024 06:34 AM
Thanks! I appreciate your help
โ12-20-2024 06:51 AM
I got response from engineering:
grpc_wait_for_shutdown_with_timeout
is actually unrelated. We have an issue open for that, but haven't yet deployed a fix, but this issue should not prevent the execution of sql magic thoughโ12-28-2024 05:28 AM
Hi @Walter_C ,
My setup is as follows:
I have modified a cell as follows (python cell, then SQL cell, then python cell):
# Databricks notebook source
df = spark.sql("SELECT 1 AS from__python_cell")
df.show()
# COMMAND ----------
# MAGIC %sql
# MAGIC SELECT 2 AS from_sql_cell
# COMMAND ----------
df = spark.sql("SELECT 3 AS from__python_cell")
df.show()
The output (the error disappeared, but nothing is displayed from SELECT from SQL):
I wonder whether the SQL was executed but simply nothing was displayed or nothing was executed.
If it is executed, do you have an idea how to display the output of the SQL statement in %sql magic command?
โ12-28-2024 08:50 AM
are you trying to see it from the interactive screen as following screenshot:
โ12-28-2024 10:53 AM
Hi @Walter_C ,
Many thanks! I was checking in terminal. After opening Interactive Window I can see the result.
Amazing!
โ12-28-2024 11:04 PM
In reality, there is nothing to do with grpc_wait_for_shutdown_with_timeout. Although we haven't yet implemented a solution, we have an open issue for it, but it shouldn't stop SQL magic from working.
Or is the "Interactive" tab where you encounter this error?
If the local Python version differs from the DBR-required Python version, it might not function.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group