Help required for executing geospetial query

dholea
New Contributor II

we have requirement to find a specific distance based on longitude and latitude. Can you please help me with the details step how we can achieve this using pyspark?

 

Thank you.

szymon_dybczak
Esteemed Contributor III

Hi @dholea ,

Databricks Runtime 17.1 Beta added native support for Spatial SQL. So for example it let's you calculate distance between coordinates.
I think you can try with ST_Distance() function.

Hi,

Thank you very much. Our data bricks version is 15.4.x-scala2.12.

Can you please guide me how to upgrade 17.1, if there is any other way to execute geospatial query, please let me know.

 

szymon_dybczak
Esteemed Contributor III

Hi @dholea ,

To update your runtime version:

1) Go to Compute (1) and click on your cluster name  (2)

szymon_dybczak_0-1755176498524.png

 

Hi @dholea ,

To update your runtime version:

1) Go to Compute (1) and click on your cluster name  (2)

szymon_dybczak_0-1755176498524.png

 

2)  Now, click Edit button

szymon_dybczak_1-1755176531590.png

3) Now you can choose required Databricks runtime

szymon_dybczak_2-1755176581008.png


Another possibility is to use H3 or Mosaic. You can read about it at below article:


GIS in Databricks — Advancing Analytics

High Scale Geospatial Processing Mosaic | Databricks Blog

2)  Now, click Edit button

szymon_dybczak_1-1755176531590.png

3) Now you can choose required Databricks runtime

szymon_dybczak_2-1755176581008.png