Hi experts, now the competition between cloud providers are fierce and brutal , I come across this post which compares Pyspark performance on databricks and Snowflakehttps://l1nk.dev/1y33h0z although the metics tested are not rigorous and exhaustive...
Hi experts ,now databricks for support for Iceberg open table format conversion yet not fully though.Databricks calls this Delta Lake UniForm (Universal Format). When tentatively Databricks can support fully open table format Iceberg, Hudi etc.Is the...
Hi experts,I recently came across an interesting discussion titled “Are enterprises moving from ‘Data Lakehouse’ to ‘Agentic Lakehouse’?” https://community.databricks.com/t5/data-engineering/are-enterprises-moving-from-quot-data-lakehouse-quot-to-quo...
Hi experts, I read an article about auto loader duplicates handling that got me bit confused. It is checkpoint that tracks what is being processed , and upon confirmation it process only the new incoming records. But, let's say I am reloading whole b...
Our power BI reports consume real-time data , and for that the only option remains is Databricks serverless,but serverrless is expensive option, how to control the costs for serverless , and any other alternatives. Thank you.
Thank you @balajij8 for reply. I agree with first part of you answer, although the link shared by you talks about different use case. As I said before, the qualifiable metrics should entirely dealt with Databricks vs Snowflake Pyspark Performance , ...
