Databricks Community

Jana · 02-15-2022

I was creating delta table from ADLS json input file. but the job was running long while creating delta table from json. Below is my cluster configuration. Is the issue related to cluster config ? Do I need to upgrade the cluster config ?The cluster ...

Jana · 03-04-2022

Increase driver memory or executor memory,? I have changed my cluster executor conf from 14 GB to 28 GB. With the changes, we were able to complete the job without an issue.

Jana · 03-03-2022

Yes, the issue was with multiline = true property. Spark is reading as whole. How to resolve the issue?

Jana · 03-01-2022

Note - Df was created with multi line true.The job was running long and slowdown the cluster performance. Can you please help me on the issueThanks

Databricks Community

User Stats

User Activity

Parsing 5 GB json file is running long on cluster

Re: Parsing 5 GB json file is running long on cluster

Re: Parsing 5 GB json file is running long on cluster

Re: Parsing 5 GB json file is running long on cluster