cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

using VSCode extension to interact with Databricks

jeremy98
New Contributor II

Hello community, 
I want to understand if it is possible to use Databricks Connect inside VSCode IDE to interact with Notebooks in local interactively like in Databricks Notebook, Is it possible? Because now I can only use the cluster and wait after the cluster executes the entire script, but If I want to use the Notebook I would to be much Interactive. 

Thanks for your answers

2 REPLIES 2

carolgrey98
Visitor

@jeremy98 wrote:

Hello community, 
I want to understand if it is possible to use Databricks Connect inside VSCode IDE to interact with Notebooks in local interactively like in Databricks Notebook, Is it possible? Because now I can only use the cluster and wait after the cluster executes the entire script, but If I want to use the Notebook I would to be much Interactive Wellstar MyChart

Thanks for your answers


Yes, it is possible to use Databricks Connect inside VSCode IDE to interact with Notebooks interactively! With the Databricks extension for Visual Studio Code, you can run and debug notebook cells one at a time or all at once, and see their results directly in the VSCode UI. This setup allows you to have a more interactive experience similar to working directly in a Databricks Notebook.

To set this up, you'll need to install the Databricks extension for VSCode and ensure you have Databricks Connect properly configured. Once everything is set up, you can run and debug your notebook cells interactively, with the Spark code running on the remote cluster and the results being sent back to your local environment

Hi, 
Thanks for your answer, actually I wrote a piece of code like:

 

 

from databricks.connect import DatabricksSession
spark = DatabricksSession.builder.getOrCreate()

 

 

 But I still have this error:

 

ImportError: cannot import name 'AnalyzeArgument' from 'pyspark.sql.udtf' (/Users/Library/Caches/pypoetry/virtualenvs/rnc-data-pipelines-Jgi6KLI_-py3.13/lib/python3.13/site-packages/pyspark/sql/udtf.py)

 


Do u know how to fix this? And For debugging the cells I need to execute the notebook in a proper way?

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group