VSCode Databricks Extension Performance
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-29-2024 01:26 AM
Hello Everyone!
I've been using the Databricks extension in VSCode for a while know and I'm syncing my repository to my Databricks workspace. In the beginning syncing files to my workspace was basically instant. But now it is starting to take a lot of time to sync files (2 to 3 minutes).
I have the feeling there is some cache buildup in VSCode, I already tried to clear the json logging files from Databricks, but this does not help. Also tried syncing manually with a different interval but it doesn't work either.
databricks sync ./my-folder/ /Users/someone@example.com/ --watch --interval 5sAnyone having the same issue or have ideas in order to get back to the usual upload time?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-07-2024 03:03 AM
Hi Kaniz,
I manage to fix this by reducing the size of the folder that I'm syncing towards Databricks. I assume when using the option '--watch' it will get somewhat slower when there are a lot of directories, subdirectories and files.
Thank you!
