how to create a nested(unflatten) json from flatten json

SailajaB
Databricks Partner

‎01-19-2022 05:29 AM

Hi ,

Is there any function in pyspark which can convert flatten json to nested json.

Ex : if we have attribute in flatten is like a_b_c : 23

then in unflatten it should be

{"a":{"b":{"c":23}}}

Thank you

cconnell
Contributor II

‎01-19-2022 05:41 AM

Your question is a little unclear. But see the lower section of this article, it might help...

https://towardsdatascience.com/json-in-databricks-and-pyspark-26437352f0e9

Hubert-Dudek
Databricks MVP

‎01-20-2022 02:44 AM

As @Chuck Connell​ said can you share more of your source json as that example is not json. Additionally flatten is usually to change something like {"status": {"A": 1,"B": 2}} to {"status.A": 1, "status.B": 2} which can be done easily with spark dataframes


My blog: https://databrickster.medium.com/
0 Kudos

SailajaB
Databricks Partner
In response to Hubert-Dudek

‎01-20-2022 02:54 AM

Hi,

Thanks for your response.

We are looking for unflattening the flatten json.

If my json contains attributes like {"status_A":2,"status_B":3,"status_C_D":5} then we should unflatten the i/p flatten json one as below

{"status":{{"A":2,"B":3},{"C":{"D":5}}}

I hope the question is clear now

Thank you

0 Kudos

jose_gonzalez
Databricks Employee
Databricks Employee
In response to SailajaB

‎02-08-2022 04:50 PM

Hi @Sailaja B​ ,

I would like to share the following example notebook https://docs.databricks.com/_static/notebooks/transform-complex-data-types-scala.html it might be able to help you.

Thank you,

--Jose

View solution in original post

0 Kudos