topic Re: Implementation of Late arriving dimension in databricks in Data Engineering

Implementation of Late arriving dimension in databricks

hare — Mon, 20 Feb 2023 06:56:59 GMT

Hi Team, Can you please suggest to me how to implement the late arriving dimension or early arriving fact with examples or any sample script for reference? I have to implement the same using pyspark.

Thanks.

Re: Implementation of Late arriving dimension in databricks

-werners- — Tue, 21 Feb 2023 12:31:49 GMT

does this help?

https://www.databricks.com/blog/2020/12/15/handling-late-arriving-dimensions-using-a-reconciliation-pattern.html

Re: Implementation of Late arriving dimension in databricks

hare — Tue, 21 Feb 2023 14:35:49 GMT

Thanks for sharing, let me check and update.

Re: Implementation of Late arriving dimension in databricks

Anonymous — Wed, 22 Feb 2023 06:54:38 GMT

Hi @Hare Krishnan

Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.

We'd love to hear from you.

Thanks!

Re: Implementation of Late arriving dimension in databricks

hare — Mon, 27 Feb 2023 04:56:39 GMT

Hi @Vidula Khanna - I will be helpful i get any sample implementation logic or script/Pseudo code like that.