- 7689 Views
- 5 replies
- 2 kudos
Split dataFrame into 1MB chunks and create a single json array with each row in chunk being an array element
Hi, I have a dataFrame that I've been able to convert into a struct with each row being a JSON object.I want the ability to split the data frame into 1MB chunks. Once I have the chunks, I would like to add all rows in each respective chunk into a sin...
- 7689 Views
- 5 replies
- 2 kudos
Latest Reply
- 2 kudos
@Tamoor Mirza :You can use the to_json method of a DataFrame to convert each chunk to a JSON string, and then append those JSON strings to a list. Here is an example code snippet that splits a DataFrame into 1MB chunks and creates a list of JSON arr...
- 2 kudos