Is it possible to run all Python (not just Spark) via Databricks cluster from local IDE?
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-17-2025 08:58 AM
Hi all,
When connecting to a Databricks cluster from a local IDE like VS Code, Spark operations already run on the cluster.
But is there a way to route all Python code — not just Spark — to run directly on the Databricks cluster as well?
Thanks!
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-18-2025 07:50 AM
Hello @tasozgurcem!
If you want your whole Python script to run on the cluster, you can either upload the file or run it as a job using the Databricks extension.
Check this out: https://docs.databricks.com/aws/en/dev-tools/vscode-ext/run
https://docs.databricks.com/aws/en/dev-tools/vscode-ext/tutorial
