Resolved! how to flatten non standard Json files in a dataframe
hello, I have a non standard Json file with a nested file structure that I have issues with. Here is an example of the json file. jsonfile= """[ { "success":true, "numRows":2, "data":{ "58251":{ "invoiceno":"58...
- 12064 Views
- 6 replies
- 8 kudos
Latest Reply
@stale stokkereit​ You can use the below function to flatten the struct fieldimport pyspark.sql.functions as F def flatten_df(nested_df): flat_cols = [c[0] for c in nested_df.dtypes if c[1][:6] != 'struct'] nested_cols = [c[0] for c in nest...
- 8 kudos