cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Jain
by New Contributor III
  • 1988 Views
  • 1 replies
  • 0 kudos

How to install GDAL on Databricks Cluster ?

I am currently using Runtime 10.4 LTS.The options available on Maven Central does not work as well as on PyPi.I am running:try:   from osgeo import gdal   except ImportError:   import gdalto validate but it throws ModuleNotFoundError: No module n...

  • 1988 Views
  • 1 replies
  • 0 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 0 kudos

@Abhishek Jain​  I can understand your issue it happens to me also multiple times so solving this issue I used to install the init script in my clusterMajor reason is that your 10X version does not support your current library so you have to find rig...

  • 0 kudos
barashe
by New Contributor II
  • 1230 Views
  • 1 replies
  • 0 kudos

Installing python modules on databricks job clusters

Different than all-purpose clusters, the databricks job new cluster configuration window does not have a "Libraries" tab, in which specific python modules could be installed. What's the best practice for installing python modules on such clusters?

  • 1230 Views
  • 1 replies
  • 0 kudos
Latest Reply
barashe
New Contributor II
  • 0 kudos

It turns out that the option exists outside of the cluster configuration scope, in the task configuration window itself - under "Advanced options" -> "Add dependent libraries".

  • 0 kudos
Labels