Hubert-Dudek
Databricks MVP
  .load("path")
  .withColumn("filePath",input_file_name())

than you can for example insert filePath to your stream sink and than get distinct value from there or use forEatch / forEatchBatch and for example insert it into spark sql table


My blog: https://databrickster.medium.com/

View solution in original post