Is it possible to run all Python (not just Spark) via Databricks cluster from local IDE?

[tasozgurcem]

Hi all,

When connecting to a Databricks cluster from a local IDE like VS Code, Spark operations already run on the cluster.

But is there a way to route all Python code — not just Spark — to run directly on the Databricks cluster as well?

Thanks!

[Advika]

Hello @tasozgurcem!

If you want your whole Python script to run on the cluster, you can either upload the file or run it as a job using the Databricks extension.
Check this out: https://docs.databricks.com/aws/en/dev-tools/vscode-ext/run
https://docs.databricks.com/aws/en/dev-tools/vscode-ext/tutorial