I'm building an Intelligent Document Processing pipeline using Databricks AI Functions (ai_parse_document and ai_extract).

I want to ingest PDF files from a Google Drive folder into a Unity Catalog Volume. I can perform a full load successfully using the Google Drive connector, but I haven't found a way to incrementally load only new or modified PDF files into the Volume.

For structured files like CSV, COPY INTO supports incremental ingestion by tracking previously loaded files. However, I couldn't find an equivalent approach for PDF files when the destination is a Unity Catalog Volume.

does anyone have some workaround for it?