karthickrs
New Contributor III

Hi ,

The extra rows could have been caused by various reasons:

  • Extra files in the directory
  • Empty or corrupt records
  • Non-JSON content being picked up on the first run

You could make sure that your input path contains only valid JSON files or you could modify your script to include only JSON files.

Karthick Ramachandran Seshadri
Data Architect | MS/MBA
Data + AI/ML/GenAI
17x Databricks Credentials