- 10631 Views
- 8 replies
- 4 kudos
Resolved! Unable to load Parquet file using Autoloader. Can someone help?
I am trying to load parquet files using Autoloader. Below is the code def autoload_to_table (data_source, source_format, table_name, checkpoint_path): query = (spark.readStream .format('cloudFiles') .option('cl...
- 10631 Views
- 8 replies
- 4 kudos
- 4 kudos
Hi again @Mayank Srivastava​ Thank you so much for getting back to us and marking the answer as best.We really appreciate your time.Wish you a great Databricks journey ahead!
- 4 kudos
- 1617 Views
- 0 replies
- 1 kudos
Databricks Autoloader Best practice
Databricks Autoloader is a popular mechanism for ingesting data/files from cloud storage into Delta; for a very high throughput source, what are the best practices to be following while scaling up an autoloader based pipeline to the tune of millions ...
- 1617 Views
- 0 replies
- 1 kudos