02-21-2023 11:01 AM
Howdy - I recently took a table FACT_TENDER and made it into a medalliona tyle TABLE to test performance since I suspected medallion would be quicker.
Key differences:
The SILVER table is quick for the medallion build, so I am at a loss here... I have tried optimziing for range joins, filtering out data, and ordering by but none of these have worked. Any thoughts? I can provide more detail here.
02-22-2023 07:30 AM
I ended up instituing true and tried PARTITIONING and PRUNING methods to boost performance, which has succeeded.
02-21-2023 11:13 PM
Hi @Jaime Tirado ,
Please refer below blog that might help you-
How to improve performance of Delta Lake MERGE INTO queries using partition pruning - Databricks
02-22-2023 02:06 AM
yes by referring this blog, you can have a much better understanding
02-22-2023 05:55 AM
I have seen this article and it is not particularly helpful to my case. I have a DELTA table so I cannot add a partition.
02-22-2023 07:30 AM
I ended up instituing true and tried PARTITIONING and PRUNING methods to boost performance, which has succeeded.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group