2 weeks ago
Hello!
Has anyone tried to host an R Shiny app using Databricks apps? It is very useful for Python apps, but it would be nice to know if anyone have any workaround to make it work with R Shiny Apps.
We also wonder about Databricks model hosting for R models. Is it even possible using some workarounds?
Appreciate any input on this!
2 weeks ago
Hi @AntonDBUser,
Please refer to these posts for R Shiny using Databricks Apps - https://docs.databricks.com/en/sparkr/shiny.html#id8, https://community.databricks.com/t5/machine-learning/host-r-shiny-app/m-p/22992
For R Models hosting, we recommend using Posit Connect.
Useful Resources
Parallelize R code using user-defined functions (UDFs) in sparklyr - Posit
Install the necessary packages and set up your credentials, it remotely access Databricks cluster through R Studio
Use sparklyr to access your Databricks cluster in RStudio.With sparklyr, you can execute distributed R code within the Spark environment in Databricks cluster.
Train a machine learning model and run predictions using spark udf
Save the trained model as an R object
Upload the R object to Databricks, save it as a pin, or store it in another suitable location
Use the vetiver package to store the model as a pin and deploy it as a REST API on Posit Connect
Other blogs
While it is technically possible to serve R models without Posit Connect, it involves several complex adjustments.
Below are the steps and options available:
Create a MLflow Run and Log the R Model and get a MLflow URI:
Use the 'crate' flavor to log the R model within an experiment
Include a list of required package versions or an renv lockfile.
Add Additional Artifacts:
Create a `pyfunc` that wraps the R model
Copy the model artifacts into a new model.
Generate `conda.yml`.
Adjust the `conda_env.yaml` to include `r-base` to add R to the container image.
Detect the version of R.
Use `rpy2` to set up the predict method and pass parameters to it.
The `pyfunc` must use `rpy2` to load and invoke the R model.
2 weeks ago
Thanks a lot for the input and reply! A lot of good material, tips and resources!
We're already using the Shiny Notebooks hosting, but we're wondering if it was possible to migrate to the new Databricks Apps? Seems like a better fit, and we also saw that it also supports Shiny, but that might be only the Python implementation of Shiny? Do you recommend to stay with Notebook hosting for now?
For the model hosting we're already crate and adding the renv when logging to mlflow, but not the Pyfunc wrapping. We'll give that a try. Do you have some code examples for the second part "Add Additional Artifacts"? That would be really helpfull!
BR,
Anton
2 weeks ago
Hi @AntonDBUser,
Do you recommend to stay with Notebook hosting for now? - Unfortunately yes as R support for Databricks Apps is a Feature request for now internally. It would be a great feature to have and is also considered for Future.
Do you have some code examples for the second part?- I am attaching a few files with some sample code, which might not work out of the box but would give a good idea on how can it be used. Hopefully it helps ๐
Kind Regards,
Parth
2 weeks ago
Great, thanks! We'll have a look and try to implement it. I'll share the results as soon as we have some ๐
2 weeks ago
Hi @AntonDBUser,
Did you get a chance to test this? Let me know if the above solution can be marked as accepted.
Thanks
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group