Hosting R apps and models

[AntonDBUser]

Hello!

Has anyone tried to host an R Shiny app using Databricks apps? It is very useful for Python apps, but it would be nice to know if anyone have any workaround to make it work with R Shiny Apps.

We also wonder about Databricks model hosting for R models. Is it even possible using some workarounds?

Appreciate any input on this! 

[parthSundarka]

Hi @AntonDBUser,

Please refer to these posts for R Shiny using Databricks Apps - https://docs.databricks.com/en/sparkr/shiny.html#id8, https://community.databricks.com/t5/machine-learning/host-r-shiny-app/m-p/22992

For R Models hosting, we recommend using Posit Connect.

• Useful Resources
  Parallelize R code using user-defined functions (UDFs) in sparklyr - Posit

  • Install the necessary packages and set up your credentials, it remotely access Databricks cluster through R Studio

  • Use sparklyr to access your Databricks cluster in RStudio.With sparklyr, you can execute distributed R code within the Spark environment in Databricks cluster.

  • Train a machine learning model and run predictions using spark udf

  • Save the trained model as an R object

  • Upload the R object to Databricks, save it as a pin, or store it in another suitable location

  • Use the vetiver package to store the model as a pin and deploy it as a REST API on Posit Connect

• Other blogs

  • Databricks and Posit announce new integrations, simplifying Lakehouse access for developers

  • From Databricks to Posit Connect: Building and Deploying a Shiny for R Application\

While it is technically possible to serve R models without Posit Connect, it involves several complex adjustments.

Below are the steps and options available:

1. Create a MLflow Run and Log the R Model and get a MLflow URI:

• Use the 'crate' flavor to log the R model within an experiment

• Include a list of required package versions or an renv lockfile.

2. Add Additional Artifacts:

• Create a `pyfunc` that wraps the R model

•   Copy the model artifacts into a new model.

•   Generate `conda.yml`.

•   Adjust the `conda_env.yaml` to include `r-base` to add R to the container image.

•   Detect the version of R.

•   Use `rpy2` to set up the predict method and pass parameters to it.

•  The `pyfunc` must use `rpy2` to load and invoke the R model.

[AntonDBUser]

Thanks a lot for the input and reply! A lot of good material, tips and resources! 

We're already using the Shiny Notebooks hosting, but we're wondering if it was possible to migrate to the new Databricks Apps? Seems like a better fit, and we also saw that it also supports Shiny, but that might be only the Python implementation of Shiny? Do you recommend to stay with Notebook hosting for now?

For the model hosting we're already crate and adding the renv when logging to mlflow, but not the Pyfunc wrapping. We'll give that a try. Do you have some code examples for the second part "Add Additional Artifacts"? That would be really helpfull!

BR,
Anton

[parthSundarka]

Hi @AntonDBUser,

Do you recommend to stay with Notebook hosting for now? - Unfortunately yes as R support for Databricks Apps is a Feature request for now internally. It would be a great feature to have and is also considered for Future.

Do you have some code examples for the second part?-  I am attaching a few files with some sample code, which might not work out of the box but would give a good idea on how can it be used. Hopefully it helps 🙂

Kind Regards,

Parth

[AntonDBUser]

Great, thanks! We'll have a look and try to implement it. I'll share the results as soon as we have some 🙂

[parthSundarka]

Hi @AntonDBUser,

Did you get a chance to test this? Let me know if the above solution can be marked as accepted.

Thanks