szymon_dybczak
Esteemed Contributor III

Hi @HW413 ,

You won't find checkpoint. COPY INTO does not use checkpoint like autoloader or spark structured streaming. 

The COPY INTO command retrieves metadata about all files in the specified source directory/prefix . So, every time you run copy into, the command first will create in-memory index of all files. You can see it yourself in SPARK UI:

szymon_dybczak_0-1758007836681.png

Then it will compare which files has been already loaded using delta log and this in-memory index of files build in previous step.