- 1533 Views
- 1 replies
- 1 kudos
Columns archive_time, commit_time, archive_time always NULL when running cloud_files_state
Am attempting to find the commit_time for a given file for a delta table using the cloud_files_state command. However, the archive_time, commit_time, and archive_time coluns are always NULL. I am running databrics runtime 11.3 and have also verified ...
- 1533 Views
- 1 replies
- 1 kudos
- 1 kudos
@Morten Stakkeland​ :The issue you are facing with the cloud_files_state command is a known limitation in Delta Lake as of the latest stable release (Delta Lake 1.0). The commit_time and protocol columns are always null, and the archive_time column i...
- 1 kudos
- 2174 Views
- 2 replies
- 3 kudos
Auto Loader: Empty fields (discovery_time, commit_time, archive_time) in cloud_files_state
Hi! Why are the fields discovery_time, commit_time, and archive_time NULL in cloud_files_state? Do I need to configure anything when creating my Auto Loader? df = spark.readStream.format("cloudFiles") \ .option("cloudFiles.format", "json") \ ...
- 2174 Views
- 2 replies
- 3 kudos
- 3 kudos
Please be sure that the DBR version is 10.5 or highercommit_time and archive_time can be null but discovery_time is set even as NOT NULL in the table definition so it is a bit strange. Please change the DBR version first.
- 3 kudos