Hi @badari_narayan ,

In VSCode, install the Databricks extension and then connect it to your existing Databricks workspace and cluster.

This setup allows you to run Databricks notebooks and scripts from your local VSCode, while using the Spark context of the connected Databricks cluster.

Explanation:

Databricks Extension in VSCode: The Databricks extension for VSCode allows you to connect to your Databricks workspace, access notebooks, and run code directly from VSCode.

Use of Spark Context: When connected to a Databricks cluster, the code you execute from VSCode uses the cluster's Spark context. This means computations and data processing are performed on the Databricks cluster, not on your local machine.