topic Re: Hosting R apps and models in Machine Learning

Hosting R apps and models

AntonDBUser — Sun, 08 Dec 2024 12:44:47 GMT

Hello!

Has anyone tried to host an R Shiny app using Databricks apps? It is very useful for Python apps, but it would be nice to know if anyone have any workaround to make it work with R Shiny Apps.

We also wonder about Databricks model hosting for R models. Is it even possible using some workarounds?

Appreciate any input on this!

Re: Hosting R apps and models

parthSundarka — Mon, 09 Dec 2024 10:36:34 GMT

Hi @AntonDBUser,

Please refer to these posts for R Shiny using Databricks Apps - https://docs.databricks.com/en/sparkr/shiny.html#id8, https://community.databricks.com/t5/machine-learning/host-r-shiny-app/m-p/22992

For R Models hosting, we recommend using Posit Connect.

Useful Resources
Parallelize R code using user-defined functions (UDFs) in sparklyr - Posit
- Install the necessary packages and set up your credentials, it remotely access Databricks cluster through R Studio
- Use sparklyr to access your Databricks cluster in RStudio.With sparklyr, you can execute distributed R code within the Spark environment in Databricks cluster.
- Train a machine learning model and run predictions using spark udf
- Save the trained model as an R object
- Upload the R object to Databricks, save it as a pin, or store it in another suitable location
- Use the vetiver package to store the model as a pin and deploy it as a REST API on Posit Connect
Other blogs
- Databricks and Posit announce new integrations, simplifying Lakehouse access for developers
- From Databricks to Posit Connect: Building and Deploying a Shiny for R Application\

While it is technically possible to serve R models without Posit Connect, it involves several complex adjustments.

Below are the steps and options available:

Create a MLflow Run and Log the R Model and get a MLflow URI:

Use the 'crate' flavor to log the R model within an experiment
Include a list of required package versions or an renv lockfile.

Add Additional Artifacts:

Create a `pyfunc` that wraps the R model
Copy the model artifacts into a new model.
Generate `conda.yml`.
Adjust the `conda_env.yaml` to include `r-base` to add R to the container image.
Detect the version of R.
Use `rpy2` to set up the predict method and pass parameters to it.
The `pyfunc` must use `rpy2` to load and invoke the R model.

Re: Hosting R apps and models

AntonDBUser — Mon, 09 Dec 2024 14:38:28 GMT

Thanks a lot for the input and reply! A lot of good material, tips and resources!

We're already using the Shiny Notebooks hosting, but we're wondering if it was possible to migrate to the new Databricks Apps? Seems like a better fit, and we also saw that it also supports Shiny, but that might be only the Python implementation of Shiny? Do you recommend to stay with Notebook hosting for now?

For the model hosting we're already crate and adding the renv when logging to mlflow, but not the Pyfunc wrapping. We'll give that a try. Do you have some code examples for the second part "Add Additional Artifacts"? That would be really helpfull!

BR,
Anton

Re: Hosting R apps and models

parthSundarka — Mon, 09 Dec 2024 15:35:10 GMT

Hi @AntonDBUser,

Do you recommend to stay with Notebook hosting for now? - Unfortunately yes as R support for Databricks Apps is a Feature request for now internally. It would be a great feature to have and is also considered for Future.

Do you have some code examples for the second part?- I am attaching a few files with some sample code, which might not work out of the box but would give a good idea on how can it be used. Hopefully it helps 🙂

Kind Regards,

Parth

Re: Hosting R apps and models

AntonDBUser — Mon, 09 Dec 2024 15:45:36 GMT

Great, thanks! We'll have a look and try to implement it. I'll share the results as soon as we have some 🙂

Re: Hosting R apps and models

parthSundarka — Wed, 11 Dec 2024 21:02:08 GMT

Hi @AntonDBUser,

Did you get a chance to test this? Let me know if the above solution can be marked as accepted.

Thanks

Re: Hosting R apps and models

AntonDBUser — Fri, 14 Feb 2025 07:42:16 GMT

Hi @parthSundarka!

We tried, but had some problems making it work. Do you have a more complete code example? That would be very helpfull!
BR, Anton