can databricks support murmur hash function
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-22-2025 01:36 PM
current project we are using murmur hash function in hadoop.we are planning for migration to databricks.can databricks support murmur hash function ?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
01-25-2025 09:03 PM
Hi @gvvishnu ,
Thanks for your question.
My understanding is that the Apache Spark `hash()` function implements the `org.apache.spark.sql.catalyst.expressions.Murmur3Hash` expression.
You can see this in the Spark source code here:
The best way however to confirm that this is equivalent to the existing 'murmur hash function in hadoop' that you have previously used is to do some tests and comparisons. Please do that testing and verify that this function returns the expected results.
Thank you.