Parent Hierarchy Queries/ Path Function /Recursive CTE's

Phani1 — Wed, 18 Jan 2023 08:05:31 GMT

Problem Statement:

We have a scenario where we get the data from the source in the format of (in actual 20 Levels and number of fields are more than 4 but for ease of understanding let’s consider below)

The actual code involved 20 levels of 4-5 fields each and with 20 joins/CTE for getting outputs.

Question:

Power BI level handling, Path Function is great and can generate the | separated lineage but, while visualizing it needs the separate column creation

But it will be overkill for the render time,

Do we have any similar in-built functions/libraries in Databricks (SQL,Pyspark, Dataframes,pandas) ??

Could you please suggest the optimized approach for this problem statement? (Consider the performance)

Kindly find the example in attachment.

Re: Parent Hierarchy Queries/ Path Function /Recursive CTE's

daniel_sahal — Thu, 19 Jan 2023 06:54:39 GMT

I don't think that we have anything similar as a built-in function. You'll need to write some custom code to achieve that.

topic Parent Hierarchy Queries/ Path Function /Recursive CTE's in Data Engineering

Parent Hierarchy Queries/ Path Function /Recursive CTE's

Re: Parent Hierarchy Queries/ Path Function /Recursive CTE's