I am using Azure Databricks Runtime (DBR) 8.3 ML with Python notebook and R cells together.
I want to use "tidyverse" and one of the dependency is rlang >= 0.4.10 and the base DBR 8.3 ML provides rlang @ 0.4.9.
I successfully upgraded the R package to rlang 0.4.11 but sessionInfo() still shows rlang 0.4.9.
Here are my command sequences:
# Initial starting environment
%r
sessionInfo()
---=== OUTPUT ===---
R version 4.0.4 (2021-02-15)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.5 LTS
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1
locale:
[1] LC_CTYPE=C.UTF-8 LC_NUMERIC=C LC_TIME=C.UTF-8
[4] LC_COLLATE=C.UTF-8 LC_MONETARY=C.UTF-8 LC_MESSAGES=C.UTF-8
[7] LC_PAPER=C.UTF-8 LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] SparkR_3.1.2 compiler_4.0.4 Rserve_1.8-7
---------------------------------------------------------------------------------------------
%r
install.packages("rlang")
---=== OUTPUT ===---
* installing *source* package ‘rlang’ ...
** package ‘rlang’ successfully unpacked and MD5 sums checked
** using staged installation
** libs
gcc -std=gnu99 -I"/usr/share/R/include" -DNDEBUG -I./lib/ -fvisibility=hidden -fpic -g -O2 -fdebug-prefix-map=/build/r-base-EkOYxp/r-base-4.0.4=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c capture.c -o capture.o
gcc -std=gnu99 -I"/usr/share/R/include" -DNDEBUG -I./lib/ -fvisibility=hidden -fpic -g -O2 -fdebug-prefix-map=/build/r-base-EkOYxp/r-base-4.0.4=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c export.c -o export.o
gcc -std=gnu99 -I"/usr/share/R/include" -DNDEBUG -I./lib/ -fvisibility=hidden -fpic -g -O2 -fdebug-prefix-map=/build/r-base-EkOYxp/r-base-4.0.4=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c internal.c -o internal.o
gcc -std=gnu99 -I"/usr/share/R/include" -DNDEBUG -I./lib/ -fvisibility=hidden -fpic -g -O2 -fdebug-prefix-map=/build/r-base-EkOYxp/r-base-4.0.4=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c lib.c -o lib.o
gcc -std=gnu99 -I"/usr/share/R/include" -DNDEBUG -I./lib/ -fvisibility=hidden -fpic -g -O2 -fdebug-prefix-map=/build/r-base-EkOYxp/r-base-4.0.4=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c version.c -o version.o
gcc -std=gnu99 -shared -L/usr/lib/R/lib -Wl,-Bsymbolic-functions -Wl,-z,relro -o rlang.so capture.o export.o internal.o lib.o version.o -L/usr/lib/R/lib -lR
installing to /databricks/spark/R/lib/00LOCK-rlang/00new/rlang/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (rlang)
Installing package into ‘/databricks/spark/R/lib’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/src/contrib/rlang_0.4.11.tar.gz'
Content type 'application/x-gzip' length 861727 bytes (841 KB)
==================================================
downloaded 841 KB
The downloaded source packages are in
‘/tmp/RtmpKRCZ1F/downloaded_packages’
---------------------------------------------------------------------------------------------
%r
installed.packages()
---=== OUTPUT ===---
Package LibPath Version
<listing truncated to get to the point>
rlang "rlang" "/databricks/spark/R/lib" "0.4.11"
<listing truncated to get to the point>
---------------------------------------------------------------------------------------------
%r
library(SparkR)
library(tidyverse)
---=== OUTPUT ===---
Error : package or namespace load failed for ‘tidyverse’ in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]):
namespace ‘rlang’ 0.4.9 is already loaded, but >= 0.4.10 is required
Some(<code style = 'font-size:10p'> Error: package or namespace load failed for ‘tidyverse’ in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]): </code>)
Error: package or namespace load failed for ‘tidyverse’ in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]):
namespace ‘rlang’ 0.4.9 is already loaded, but >= 0.4.10 is required
---------------------------------------------------------------------------------------------
%r
sessionInfo()
---=== OUTPUT ===---
R version 4.0.4 (2021-02-15)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.5 LTS
Matrix products: default
BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1
locale:
[1] LC_CTYPE=C.UTF-8 LC_NUMERIC=C LC_TIME=C.UTF-8
[4] LC_COLLATE=C.UTF-8 LC_MONETARY=C.UTF-8 LC_MESSAGES=C.UTF-8
[7] LC_PAPER=C.UTF-8 LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C LC_MEASUREMENT=C.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] SparkR_3.1.2
loaded via a namespace (and not attached):
[1] compiler_4.0.4 hwriter_1.3.2 tools_4.0.4 htmltools_0.5.0
[5] TeachingDemos_2.10 Rserve_1.8-7 digest_0.6.27 rlang_0.4.9
[9] hwriterPlus_1.0-3
Has anyone ran into this issue and provide some clue as to how to update the rlang package to be >- 0.4.10 so "tidyverse" can load?
