02-01-2022 03:25 AM
Hello DataBricks,
Struggling with a workflow issue and wondering if anyone can help. I am developing my project in R and sometimes Python locally on my laptop, and committing the files to a git repo. I can then clone that repo in databricks, and *see* the files, but I can't run them. Is there any reccomended workflow or way to do this? The picture below is an example of how my R script appears in Databricks once I have imported the repository. I can't attach to a cluster, and I can't run any of the lines/commands.
Thanks, James
02-01-2022 07:08 AM
This is separate script which than need to be run from notebook (or job). I am not using R but in Python and Scala it works the same. In Python I am just importing it in notebook ("from folder_structure import myClass") in R probably similar. There are also other ways as well; for example develop in notebooks or use RStudio on databricks https://docs.databricks.com/spark/latest/sparkr/rstudio.html
02-01-2022 07:02 AM
Hi @James Smith ! My name is Kaniz, and I'm the technical moderator here. Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question first. Or else I will get back to you soon. Thanks.
02-01-2022 07:08 AM
This is separate script which than need to be run from notebook (or job). I am not using R but in Python and Scala it works the same. In Python I am just importing it in notebook ("from folder_structure import myClass") in R probably similar. There are also other ways as well; for example develop in notebooks or use RStudio on databricks https://docs.databricks.com/spark/latest/sparkr/rstudio.html
02-04-2022 12:41 PM
Hi @James Smith , I would be very happy to know whether @Hubert Dudek 's response answer your query?
02-04-2022 01:02 PM
Kind of thanks. It tells me how to 'work around' it. But I think it's a shame that a notebook can't recognise/parse a script to allow interactive running of commands, rather than having to run or source the whole script.
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.