Databricks Community

DB_Learner_3

Description:As a data engineer, I need to implement an automated process to ingest data from multiple files in a subdirectory and create corresponding bronze tables. This process should handle full file refreshes and consider strategies to limit the ...

DB_Learner_3

Thank you will try that; this is how the files are named and directory by each source.

DB_Learner_3

We are trying an ingestion process to bronze table from a volume, and in that volume if multiple CSV files are there it need to loop through each file as a full refresh mode (truncate the old file and load the new file) , if we are using autoloader o...

Databricks Community

User Stats

User Activity

Help needed with automated file ingestion process for full refresh mode using autoloader preferable

Re: Help needed with automated file ingestion process for full refresh mode using autoloader prefera

Re: Help needed with automated file ingestion process for full refresh mode using autoloader prefera