can databricks support murmur hash function

gvvishnu
New Contributor

current project we are using murmur hash function in hadoop.we are planning for migration to databricks.can databricks support murmur hash function ? 

brockb
Databricks Employee
Databricks Employee

Hi @gvvishnu ,

Thanks for your question.

My understanding is that the Apache Spark `hash()` function implements the `org.apache.spark.sql.catalyst.expressions.Murmur3Hash` expression.

You can see this in the Spark source code here:

https://github.com/apache/spark/blob/master/sql/core/src/test/resources/sql-functions/sql-expression...

The best way however to confirm that this is equivalent to the existing 'murmur hash function in hadoop' that you have previously used is to do some tests and comparisons. Please do that testing and verify that this function returns the expected results.

Thank you.