topic Re: How to convert lat/long to geohash in databricks using geohashTools R library in Data Engineering

How to convert lat/long to geohash in databricks using geohashTools R library

Jeff1 — Mon, 07 Mar 2022 19:53:52 GMT

I continues to receive a parsing error when attempting to convert lat/long data to a geohash in data bricks . I've tried two coding methods in R and get the same error.

library(geohashTools)

Method #1

my_tbl$geo_hash <- gh_encode(my_tbl$Latitude, my_tbl$Longitude, precision = 9L)

Method #2

my_tbl <- my_tbl %>%

mutate(geo_hash = gh_encode(Latitude, Longitude, precision = 9L))

Any suggestions?

Jeff

Re: How to convert lat/long to geohash in databricks using geohashTools R library

Hubert-Dudek — Tue, 08 Mar 2022 10:50:22 GMT

Please install required library in cluster settings tab -> libraries, than install new and install it from CRAN. I tested and after that geoHashTools worked without problem (second screenshot)

Re: How to convert lat/long to geohash in databricks using geohashTools R library

Jeff1 — Tue, 08 Mar 2022 14:07:12 GMT

Whmmmmm. I ran installed.packages() from the command line and its shows the geohashTools package libpath as "/databricks/spark/R/lib" (version 0.3.1) so I am assuming I've got it loaded. So then I ran the following test:

lat <- c(24.123, 45.123)

long <- c(145.123, 130.124)

test <- gh_encode(lat, long, precision = 9L)

test

[1] "xhrlnr7qd" "ybh89n3ks"

So the command works but the following code

# add geohash

my_tbl <- my_tbl %>%

mutate(geo_hash = gh_encode(Latitude, Longitude, precision = 9L)

results in the following error message:

Error in Use("escape") : Error in UseMethod("escape") :

no application method for 'escape' applied to an object of class "c('my_tbl', 'tbl_lazy', 'tbl')"

I'm to new to databricks to have any clue what the error is trying to tell me.

Any other suggestions?

----------------------

I just couldn't get the R gh_encode function to work on the Spark df so I ran the collect() function and then re-ran the mutate and gh_encode functions on the R data.frame and converted back to a Spark dataframe. That appears to have worked. However not the optimal solution (need to work in the Spark dataframe) but worked.

Re: How to convert lat/long to geohash in databricks using geohashTools R library

Jeff1 — Thu, 17 Mar 2022 14:05:31 GMT

The problem was I was trying to run the gh_encode function on a Spark dataframe. I needed to collect the date into a R dataframe then run the function.