topic Re: Convert table in nested JSON in Data Engineering

Convert table in nested JSON

spaz — Sun, 13 Feb 2022 21:03:22 GMT

What is the easiest way to convert a table to a nested JSON?

Re: Convert table in nested JSON

AmanSehgal — Sun, 13 Feb 2022 22:02:36 GMT

@Sergio Paz could you please provide an example of your input table and expected output JSON file?

Re: Convert table in nested JSON

-werners- — Mon, 14 Feb 2022 10:12:27 GMT

As Spark can handle nested columns, I would first construct the nested structure in spark (as from spark 3.1.1 there is the excellent column.withField method with which you can create your structure.

Finally write it to json.

That seems to be the easiest way, but your case might be more complex, that is hard to say without some more info.

Re: Convert table in nested JSON

Anonymous — Mon, 07 Mar 2022 00:08:12 GMT

@Sergio Paz - How's it going? Are you able to give us more information?

Re: Convert table in nested JSON

jose_gonzalez — Tue, 22 Mar 2022 21:22:39 GMT

Hi @Sergio Paz ,

Just a friendly follow-up. Could you provide more details on your use case? please share your code snippet, so we can help you.

Re: Convert table in nested JSON

spaz — Tue, 12 Apr 2022 02:52:19 GMT

{

"accounts":[

"12516898"

"deals":[

{

"dealId":"4897143",

"promotionId":"AVC84897143",

"conditions":{

"paymentMethod":null,

"simulationDateTime":{

"startDateTime":"2022-03-16",

"endDateTime":"2022-04-30"

"scaledLineItem":{

"sharedMinimumQuantity":true,

"skus":[

"000000000000000031"

"crossDiscount":true,

"ranges":[

{

"index":"1",

"from":1,

"to":200

}

]

}

"output":{

"lineItemScaledDiscount":{

"ranges":[

{

"index":"1",

"skus":[

"000000000000000031"

"type":"%",

"discount":5.5,

"maxQuantity":null,

"proportion":null,

"fixed":true

}

]

}

]

}