Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
07-02-2021 09:18 AM
This is due to the when users run their R scripts on Rstudio, the R session is not shut down gracefully.
Databricks is working on handle the R session better and removed the limit.
As a workaround, you can create and run below init script to increase the limit:
%scala
val initScriptContent = s"""
|#!/bin/bash
|cat > /databricks/common/conf/rbackend_limit.conf << EOL
|{
| databricks.daemon.driver.maxNumRBackendsPerDriver = <value>
|}
|EOL
""".stripMargin
dbutils.fs.put("dbfs:/<path>/set_rbackend.sh",initScriptContent, true)