cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Unable to install R geospatial libraries raster, terra, sf, ncdf4, etc

ArturoNuor
New Contributor III

When trying to install any of this R libraries from a cmd cell/block on a notebook, or from the UI in the cluster I receive the same error,seeming that are unable to install dependecies.

Warning in utils::install.packages(pkgs, ...) :

installation of package โ€˜*****โ€™ had non-zero exit status

Can anyone help me with this?

1 ACCEPTED SOLUTION

Accepted Solutions

ArturoNuor
New Contributor III

For the next soul looking for an answer, I managed to solve the issue with the next 2 Init scripts, it gets tricky in the apt or apt-get, that was the issue, sometimes it did update, sometimes it didn't, making it possible to find libmysqlclient21.

1)

%sh

sudo rm -r /var/lib/apt/lists/* 

sudo apt clean && 

  sudo apt update --fix-missing -y &&

  sudo apt install -y libmysqlclient21

sudo apt install -y gdal-bin

2)

%sh

sudo apt --fix-missing -y update --fix-missing

sudo apt-get install -y libudunits2-dev libproj-dev libgeos-dev libgdal-dev

View solution in original post

3 REPLIES 3

Debayan
Databricks Employee
Databricks Employee

Hi, receiving this message means that the package was not installed. There can be different reasons why this happens, which means there are also different ways to fix this. The most common cause when installing the required package is the existence of unresolved dependencies. Could you please check the same?

Please tag @Debayanโ€‹ with your next comment so that I will get notified. Thank you!

ArturoNuor
New Contributor III

Hi @Debayan Mukherjeeโ€‹ thanks for your reply, here are my R/Databricks details and the 3 main libraries and errors i am encountering:

      

platform    x86_64-pc-linux-gnu          

version.string R version 4.1.3 (2022-03-10)

Runtime

11.3.x-scala2.12

11.3 LTS (includes Apache Spark 3.3.0, Scala 2.12)

gdal

%r

install.packages("gdal", repos = "http://cran.us.r-project.org")

Installing package into โ€˜/local_disk0/.ephemeral_nfs/envs/rEnv-93f5aa20-8204-42c9-b813-cda1cb27ae08โ€™

(as โ€˜libโ€™ is unspecified)

Warning: package โ€˜gdalโ€™ is not available for this version of R

A version of this package for your version of R might be available elsewhere,

see the ideas at

https://cran.r-project.org/doc/manuals/r-patched/R-admin.html#Installing-packages

terra

%r

install.packages("terra", repos = "http://cran.us.r-project.org")

* installing *source* package โ€˜terraโ€™ ...

** package โ€˜terraโ€™ successfully unpacked and MD5 sums checked

** using staged installation

configure: CC: gcc

configure: CXX: g++ -std=gnu++14

checking for gdal-config... no

no

configure: error: gdal-config not found or not executable.

ERROR: configuration failed for package โ€˜terraโ€™

* removing โ€˜/local_disk0/.ephemeral_nfs/envs/rEnv-93f5aa20-8204-42c9-b813-cda1cb27ae08/terraโ€™

Installing package into โ€˜/local_disk0/.ephemeral_nfs/envs/rEnv-93f5aa20-8204-42c9-b813-cda1cb27ae08โ€™

(as โ€˜libโ€™ is unspecified)

trying URL 'http://cran.us.r-project.org/src/contrib/terra_1.7-29.tar.gz'

Content type 'application/x-gzip' length 806064 bytes (787 KB)

==================================================

downloaded 787 KB

Warning in utils::install.packages(pkgs, ...) :

 installation of package โ€˜terraโ€™ had non-zero exit status

The downloaded source packages are in

โ€˜/tmp/RtmpPqEEXP/downloaded_packagesโ€™

raster

%r

install.packages("raster", repos = "http://cran.us.r-project.org")

* installing *source* package โ€˜terraโ€™ ...

** package โ€˜terraโ€™ successfully unpacked and MD5 sums checked

** using staged installation

configure: CC: gcc

configure: CXX: g++ -std=gnu++14

checking for gdal-config... no

no

configure: error: gdal-config not found or not executable.

ERROR: configuration failed for package โ€˜terraโ€™

* removing โ€˜/local_disk0/.ephemeral_nfs/envs/rEnv-93f5aa20-8204-42c9-b813-cda1cb27ae08/terraโ€™

ERROR: dependency โ€˜terraโ€™ is not available for package โ€˜rasterโ€™

* removing โ€˜/local_disk0/.ephemeral_nfs/envs/rEnv-93f5aa20-8204-42c9-b813-cda1cb27ae08/rasterโ€™

Installing package into โ€˜/local_disk0/.ephemeral_nfs/envs/rEnv-93f5aa20-8204-42c9-b813-cda1cb27ae08โ€™

(as โ€˜libโ€™ is unspecified)

also installing the dependency โ€˜terraโ€™

trying URL 'http://cran.us.r-project.org/src/contrib/terra_1.7-29.tar.gz'

Content type 'application/x-gzip' length 806064 bytes (787 KB)

==================================================

downloaded 787 KB

ArturoNuor
New Contributor III

For the next soul looking for an answer, I managed to solve the issue with the next 2 Init scripts, it gets tricky in the apt or apt-get, that was the issue, sometimes it did update, sometimes it didn't, making it possible to find libmysqlclient21.

1)

%sh

sudo rm -r /var/lib/apt/lists/* 

sudo apt clean && 

  sudo apt update --fix-missing -y &&

  sudo apt install -y libmysqlclient21

sudo apt install -y gdal-bin

2)

%sh

sudo apt --fix-missing -y update --fix-missing

sudo apt-get install -y libudunits2-dev libproj-dev libgeos-dev libgdal-dev

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group