I have 100+ tables , with CDC I’m getting files on GCS bucket every 15mins or some random time based on source changes.I have enabled file arrival triggers for each tables.is this good approach or consolidating tables in one job to have one trigger?
@Sneeze7432 did you solve ?File arrival group the files when it executes, I verified this with Databricks team.you may encounter Multiple source matched error during MERGE operations. to overcome, It’s better to APPEND only into to Raw/bronze layer, ...
