Hi patacoing,

How are you doing today?, As per my understanding, The structure you described in your S3 data lake sounds more like a "pre-bronze" layer, since the files are in mixed formats (JSON, CSV, text, binary), which makes it tricky to process them directly with Spark in a uniform way. In Databricks, your bronze layer is usually where data becomes readable and queryable—often standardized into Delta format. A good approach is to use Autoloader to ingest each file type separately by setting the correct format (like .format("cloudFiles").option("cloudFiles.format", "json") and so on), and then write them into a bronze Delta table with consistent schema. If formats are very inconsistent or unknown, you could even store the raw content in a Delta table using a binary column along with a metadata map to track file info. This lets you store everything safely and do transformations later in silver/gold layers. So yes, Autoloader is definitely still relevant—you just need to process one format at a time or wrap each file’s raw content smartly. Let me know if you'd like a sample bronze setup based on your structure!

Regards,

Brahma