Databricks Community

Hi,I have input files in S3 with below structure./mnt/<mount_name>/test/<company_id>/sales/file_1.json/mnt/<mount_name>/test/<company_id>/sales/file_2.json/mnt/<mount_name>/test/<company_id>/sales/file_<n>.jsonNumber of companies = 15Number of files ...

Hi,I need to process nearly 30 files from different locations and insert records to RDS. I am using multi-threading to process these files parallelly like below. Test data:             I have configuration like below based on column 4: If column 4=0:...

Hi,I need to process nearly 30 files from different locations and insert records to RDS.I am using multi-threading to process these files parallelly like below. def process_files(file_path):    <process files here>    1. Find bad records based on fie...

Hi,I have data in file like belowI have different types of row in my input file, column number 8 defines the type of the record.In the above file we have 4 types of records 00 to 03My requirement is:There will be multiple files in the source path, ea...