Testing Databricks Auto Loader File Notification (File Event) in Public Preview - Spark Termination

dawn-dot-py
Databricks Partner

‎08-06-2025 09:58 PM

I tried to test the Databricks Auto Loader file notification (file event) feature, which is currently in public preview, using a notebook for work purposes. However, when I ran display(df), Spark terminated and threw the error shown in the attached image.

Is the file event mode in the public preview phase currently not operational? I am still learning about Databricks, so I am asking here for help.

dawndotpy_0-1754542620496.png

0 Kudos

Advika
Community Manager
Community Manager

‎08-08-2025 04:04 AM

Hello @dawn-dot-py!

Auto Loader’s managed file events are indeed in Public Preview, but they’re available to allowlisted workspaces. The error you encountered means your workspace hasn’t been enrolled in the preview, which is expected unless you’ve been granted access.
To use this feature, you can reach out to Databricks Support or your account team to request enrollment. Until then, you can set cloudFiles.useManagedFileEvents=false.

https://docs.databricks.com/aws/en/ingestion/cloud-object-storage/auto-loader/file-notification-mode...

View solution in original post