topic Re: RStudio on Databricks user experience in Data Engineering

RStudio on Databricks user experience

J15S — Wed, 21 Dec 2022 14:49:08 GMT

Is anybody actually using the RStudio app integration on Databricks? I'm surprised to find so little discussion in this forum.

My team has been using it for about 3 months and it seems under-developed.

1) No automated backup, you have to do it yourself Rstudio-automated-backup - Databricks

2) Spark connection gets dropped on a regular basis

3) Can't access repos in the `/Workspaces/Repos/` path though I can see it there on the web terminal.

Is everyone basically using notebooks?

Re: RStudio on Databricks user experience

jonathan-dufaul — Fri, 23 Dec 2022 17:53:36 GMT

I tried it but I too felt it was clunky for the above reasons (3 being the fatal one). In addition:

can't use it on a cluster that autoterminates/actually had to have a dedicated cluster for each user I believe
the login process for the open source version is bad (allowing one user and having to share the credentials)
persisting the workspace at all is a pain

All of this work is to avoid using a notebook that is tightly integrated with databricks. The reward isn't worth the effort.

Re: RStudio on Databricks user experience

J15S — Tue, 03 Jan 2023 13:17:58 GMT

@Jonathan Dufault Thanks for the response, and glad I'm not alone. My problem (and this is probably just a preference thing) is that the 'reward' of using a full-fledged IDE is huge, compared to bouncing between notebooks in multiple tabs. The integrations are all (mostly?) Databricks specific and my team hasn't fully committed yet to try and keep some platform independence so we're not getting much benefit there.

Re: RStudio on Databricks user experience

jonathan-dufaul — Tue, 03 Jan 2023 14:23:59 GMT

There's a submitted idea on the azure databricks feedback page https://feedback.azure.com/d365community/idea/4a538c5f-f124-ec11-b6e6-000d3a4f0da0 if you wanted to upvote.

Re: RStudio on Databricks user experience

J15S — Tue, 03 Jan 2023 14:38:51 GMT

wow ... the idea was submitted 4 years ago 😞 i guess i won't get my hopes up for a quick change