Re: GeoPandas Insall

tomos_phillips1 · ‎04-05-2024

hi,

I cannot install geopandas in my notebook, ive tried all different forms of generic fix, pip installs etc but always get this error:

CalledProcessError: Command 'pip --disable-pip-version-check install geopandas' returned non-zero exit status 1.

--------------------------------------------------------------------------- CalledProcessError Traceback (most recent call last) File <command-3215529319294224>, line 3 1 get_ipython().run_line_magic('pip', 'install folium') 2 get_ipython().run_line_magic('pip', 'install shapely') ----> 3 get_ipython().run_line_magic('pip', 'install geopandas') 4 get_ipython().run_line_magic('pip', 'install geopy') 5 get_ipython().run_line_magic('pip', 'install rtree') File /databricks/python/lib/python3.10/site-packages/IPython/core/interactiveshell.py:2369, in InteractiveShell.run_line_magic(self, magic_name, line, _stack_depth) 2367 kwargs['local_ns'] = self.get_local_scope(stack_depth) 2368 with self.builtin_trap: -> 2369 result = fn(*args, **kwargs) 2371 # The code below prevents the output from being displayed 2372 # when using magics with decodator @output_can_be_silenced 2373 # when the last Python token in the expression is a ';'. 2374 if getattr(fn, magic.MAGIC_OUTPUT_CAN_BE_SILENCED, False😞 File /databricks/python_shell/dbruntime/PipMagicOverrides.py:34, in PipMagicOverrides.pip(self, line) 32 @line_magic 33 def pip(self, line): ---> 34 self.pipMagicHandler.runCmd("pip", line) File /databricks/python_shell/dbruntime/PipMagicOverrides.py:60, in PipMagicHandler.runCmd(self, magicCmd, line) 58 print(PYTHON_RESTART_WARNING) 59 if parsedResult.rewrittenCommand(): ---> 60 self.executePipCommand(parsedResult) 61 envManager.postExecute(parsedResult) 62 if parsedResult.isMutation(): 63 # double print this output is at the end so it is more 64 # likely to be seen File /databricks/python_shell/dbruntime/PipMagicOverrides.py:123, in PipMagicHandler.executePipCommand(self, result) 121 sys.stdout.flush() 122 if returncode != 0: --> 123 raise subprocess.CalledProcessError(returncode, origCmd) 124 finally: 125 end = time.time() CalledProcessError: Command 'pip --disable-pip-version-check install geopandas' returned non-zero exit status 1.

note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error × Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output. note: This error originates from a subprocess, and is likely not a problem with pip.

Can anyone help me with this issue?

Thanks.

shan_chandra · ‎04-05-2024

@tomos_phillips1 - can you please try install geopandas on a single node cluster. (driver only) ?

%pip install geopandas

tomos_phillips1 · ‎04-06-2024

@shan_chandra - I do not have permission in my org to modify any cluster properties etc. Do you know of another way around this? Our internal IT team also do not have a fix for this.

shan_chandra · ‎04-10-2024

@tomos_phillips1 - can you please raise a support ticket with Databricks support team to triage this further?

vbvasa · ‎04-23-2024

@shan_chandra - Doesn't work for me in Driver Only Cluster as well

vbvasa · ‎04-23-2024

@tomos_phillips1 @shan_chandra
Got the below init script from Databricks Support. Worked for us in Databricks AWS Env.

dbutils.fs.put("/databricks/scripts/libinstall.sh","""
#!/bin/bash
sudo rm -r /var/lib/apt/lists/*
sudo apt clean &&
sudo apt update --fix-missing -y &&
sudo apt install -y libmysqlclient21
sudo apt install -y libgdal-dev
""", True)

brian999 · ‎09-06-2024

Thanks so much for this it worked for me!

brian999 · ‎09-06-2024

I have been having this issue also. Conda may well be better, but how do you use conda in databricks? From what I can see the only ways are to either use runtime 7.3 or below or use completely new type of container. Those are not feasible.

shan_chandra · ‎09-06-2024

@brian999 - Conda is subjected to commercial licensing. Referenced here: https://docs.databricks.com/en/archive/legacy/conda.html

brian999 · ‎09-06-2024

As i said in my comment and it says on the page you just sent:

Important

%conda commands are deprecated, and are supported only for Databricks Runtime 7.3 LTS ML. Databricks recommends using %pip for managing notebook-scoped libraries. If you require Python libraries that can only be installed using conda, you can use conda-based docker containers to pre-install the libraries you need.

This is not at all a feasible way to use conda.