how to create a nested(unflatten) json from flatten json

[SailajaB]

Hi ,

Is there any function in pyspark which can convert flatten json to nested json.

Ex : if we have attribute in flatten is like a_b_c : 23

then in unflatten it should be

{"a":{"b":{"c":23}}}

Thank you

[cconnell]

Your question is a little unclear. But see the lower section of this article, it might help...

https://towardsdatascience.com/json-in-databricks-and-pyspark-26437352f0e9

[Hubert-Dudek]

As @Chuck Connell​ said can you share more of your source json as that example is not json. Additionally flatten is usually to change something like {"status": {"A": 1,"B": 2}} to {"status.A": 1, "status.B": 2} which can be done easily with spark dataframes

[SailajaB]

Hi,

Thanks for your response.

We are looking for unflattening the flatten json.

If my json contains attributes like {"status_A":2,"status_B":3,"status_C_D":5} then we should unflatten the i/p flatten json one as below

{"status":{{"A":2,"B":3},{"C":{"D":5}}}

I hope the question is clear now

Thank you

[jose_gonzalez · Accepted solution]

Hi @Sailaja B​ ,

I would like to share the following example notebook https://docs.databricks.com/_static/notebooks/transform-complex-data-types-scala.html it might be able to help you.

Thank you,

--Jose