GeoPandas Insall

[tomos_phillips1]

hi,

I cannot install geopandas in my notebook, ive tried all different forms of generic fix, pip installs etc but always get this error:

CalledProcessError: Command 'pip --disable-pip-version-check install geopandas' returned non-zero exit status 1.

--------------------------------------------------------------------------- CalledProcessError Traceback (most recent call last) File <command-3215529319294224>, line 3 1 get_ipython().run_line_magic('pip', 'install folium') 2 get_ipython().run_line_magic('pip', 'install shapely') ----> 3 get_ipython().run_line_magic('pip', 'install geopandas') 4 get_ipython().run_line_magic('pip', 'install geopy') 5 get_ipython().run_line_magic('pip', 'install rtree') File /databricks/python/lib/python3.10/site-packages/IPython/core/interactiveshell.py:2369, in InteractiveShell.run_line_magic(self, magic_name, line, _stack_depth) 2367 kwargs['local_ns'] = self.get_local_scope(stack_depth) 2368 with self.builtin_trap: -> 2369 result = fn(*args, **kwargs) 2371 # The code below prevents the output from being displayed 2372 # when using magics with decodator @output_can_be_silenced 2373 # when the last Python token in the expression is a ';'. 2374 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False😞 File /databricks/python_shell/dbruntime/PipMagicOverrides.py:34, in PipMagicOverrides.pip(self, line) 32 @line_magic 33 def pip(self, line): ---> 34 self.pipMagicHandler.runCmd("pip", line) File /databricks/python_shell/dbruntime/PipMagicOverrides.py:60, in PipMagicHandler.runCmd(self, magicCmd, line) 58 print(PYTHON_RESTART_WARNING) 59 if parsedResult.rewrittenCommand(): ---> 60 self.executePipCommand(parsedResult) 61 envManager.postExecute(parsedResult) 62 if parsedResult.isMutation(): 63 # double print this output is at the end so it is more 64 # likely to be seen File /databricks/python_shell/dbruntime/PipMagicOverrides.py:123, in PipMagicHandler.executePipCommand(self, result) 121 sys.stdout.flush() 122 if returncode != 0: --> 123 raise subprocess.CalledProcessError(returncode, origCmd) 124 finally: 125 end = time.time() CalledProcessError: Command 'pip --disable-pip-version-check install geopandas' returned non-zero exit status 1.

note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip.

Can anyone help me with this issue?

Thanks. 

[shan_chandra]

@tomos_phillips1 - can you please try install geopandas on a single node cluster. (driver only) ?

%pip install geopandas

[tomos_phillips1]

@shan_chandra - I do not have permission in my org to modify any cluster properties etc. Do you know of another way around this? Our internal IT team also do not have a fix for this. 

[shan_chandra]

@tomos_phillips1 - can you please raise a support ticket with Databricks support team to triage this further?

[vbvasa]

@shan_chandra - Doesn't work for me in Driver Only Cluster as well

[vbvasa]

@tomos_phillips1 @shan_chandra 
Got the below init script from Databricks Support. Worked for us in Databricks AWS Env.

dbutils.fs.put("/databricks/scripts/libinstall.sh","""
#!/bin/bash
sudo rm -r /var/lib/apt/lists/* 
sudo apt clean && 
sudo apt update --fix-missing -y &&
sudo apt install -y libmysqlclient21
sudo apt install -y libgdal-dev
""", True)

[brian999]

Thanks so much for this it worked for me!

[Kaniz_Fatma]

Hi @vbvasa, 

• The error message indicates that a GDAL API version must be specified. You can address this by providing a path to gdal-config using a GDAL_CONFIG environment variable or ...¹.
• To set the GDAL_CONFIG environment variable, you can follow these steps:
  • Locate the path to your GDAL installation (usually in the bin directory).
  • Set the GDAL_CONFIG environment variable to point to the gdal-config executable. For example:

    export GDAL_CONFIG=/path/to/gdal-config
    

• Alternatively, you can set the GDAL_VERSION environment variable to the desired GDAL version.
• Consider using Conda instead of pip to install GeoPandas. Conda handles package dependencies more effectively and can help avoid version conflicts.
• Create a new environment and install GeoPandas within it:

  conda create -n geo-env -c conda-forge geopandas
  

• If this fails, try updating Conda and then retry the installation
• Sometimes, manually installing dependencies can resolve issues.
• Follow these steps:
  • Install the required dependencies one by one:

    pip install cython
    pip install shapely
    pip install fiona
    pip install pyproj
    pip install rtree
    

  • After installing the dependencies, try installing GeoPandas again:

    pip install geopandas

• Remember to activate the appropriate environment (if using Conda) before attempting the installation.
  
  Hopefully, one of these approaches will help you successfully install GeoPandas! 😊🌐

[brian999]

I have been having this issue also. Conda may well be better, but how do you use conda in databricks? From what I can see the only ways are to either use runtime 7.3 or below or use completely new type of container. Those are not feasible.

[shan_chandra]

@brian999 - Conda is subjected to commercial licensing. Referenced here:  https://docs.databricks.com/en/archive/legacy/conda.html

[brian999]

As i said in my comment and it says on the page you just sent:

Important

%conda commands are deprecated, and are supported only for Databricks Runtime 7.3 LTS ML. Databricks recommends using %pip for managing notebook-scoped libraries. If you require Python libraries that can only be installed using conda, you can use conda-based docker containers to pre-install the libraries you need.

This is not at all a feasible way to use conda.