topic How to plan for 2 million json files ingestion in production in Data Engineering https://community.databricks.com/t5/data-engineering/how-to-plan-for-2-million-json-files-ingestion-in-production/m-p/36070#M26039 <P>We are planning to load 2 million plus jsons into prod. How should we plan for any intermittent failures and complete loads without performance issues.</P> Wed, 28 Jun 2023 23:53:07 GMT Yogybricks 2023-06-28T23:53:07Z How to plan for 2 million json files ingestion in production https://community.databricks.com/t5/data-engineering/how-to-plan-for-2-million-json-files-ingestion-in-production/m-p/36070#M26039 <P>We are planning to load 2 million plus jsons into prod. How should we plan for any intermittent failures and complete loads without performance issues.</P> Wed, 28 Jun 2023 23:53:07 GMT https://community.databricks.com/t5/data-engineering/how-to-plan-for-2-million-json-files-ingestion-in-production/m-p/36070#M26039 Yogybricks 2023-06-28T23:53:07Z