04-05-2024 05:49 AM
hi,
I cannot install geopandas in my notebook, ive tried all different forms of generic fix, pip installs etc but always get this error:
04-05-2024 10:23 AM
@tomos_phillips1 - can you please try install geopandas on a single node cluster. (driver only) ?
%pip install geopandas
04-06-2024 04:14 AM
@shan_chandra - I do not have permission in my org to modify any cluster properties etc. Do you know of another way around this? Our internal IT team also do not have a fix for this.
04-10-2024 01:11 PM
@tomos_phillips1 - can you please raise a support ticket with Databricks support team to triage this further?
04-23-2024 04:08 AM
04-23-2024 07:11 AM
@tomos_phillips1 @shan_chandra
Got the below init script from Databricks Support. Worked for us in Databricks AWS Env.
dbutils.fs.put("/databricks/scripts/libinstall.sh","""
#!/bin/bash
sudo rm -r /var/lib/apt/lists/*
sudo apt clean &&
sudo apt update --fix-missing -y &&
sudo apt install -y libmysqlclient21
sudo apt install -y libgdal-dev
""", True)
4 weeks ago
Thanks so much for this it worked for me!
04-29-2024 03:14 AM
Hi @vbvasa,
gdal-config
using a GDAL_CONFIG
environment variable or ...
1.GDAL_CONFIG
environment variable, you can follow these steps:
bin
directory).GDAL_CONFIG
environment variable to point to the gdal-config
executable. For example:
export GDAL_CONFIG=/path/to/gdal-config
GDAL_VERSION
environment variable to the desired GDAL version.conda create -n geo-env -c conda-forge geopandas
pip install cython
pip install shapely
pip install fiona
pip install pyproj
pip install rtree
pip install geopandas
4 weeks ago
I have been having this issue also. Conda may well be better, but how do you use conda in databricks? From what I can see the only ways are to either use runtime 7.3 or below or use completely new type of container. Those are not feasible.
4 weeks ago
@brian999 - Conda is subjected to commercial licensing. Referenced here: https://docs.databricks.com/en/archive/legacy/conda.html
4 weeks ago - last edited 4 weeks ago
As i said in my comment and it says on the page you just sent:
Important
%conda commands are deprecated, and are supported only for Databricks Runtime 7.3 LTS ML. Databricks recommends using %pip for managing notebook-scoped libraries. If you require Python libraries that can only be installed using conda, you can use conda-based docker containers to pre-install the libraries you need.
This is not at all a feasible way to use conda.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group