cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

MRTN
by New Contributor III
  • 751 Views
  • 1 replies
  • 1 kudos

Columns archive_time, commit_time, archive_time always NULL when running cloud_files_state

Am attempting to find the commit_time for a given file for a delta table using the cloud_files_state command. However, the archive_time, commit_time, and archive_time coluns are always NULL. I am running databrics runtime 11.3 and have also verified ...

cloud_files_state
  • 751 Views
  • 1 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

@Morten Stakkeland​ :The issue you are facing with the cloud_files_state command is a known limitation in Delta Lake as of the latest stable release (Delta Lake 1.0). The commit_time and protocol columns are always null, and the archive_time column i...

  • 1 kudos
Benji0934
by New Contributor II
  • 980 Views
  • 2 replies
  • 3 kudos

Auto Loader: Empty fields (discovery_time, commit_time, archive_time) in cloud_files_state

Hi! Why are the fields discovery_time, commit_time, and archive_time NULL in cloud_files_state? Do I need to configure anything when creating my Auto Loader? df = spark.readStream.format("cloudFiles") \ .option("cloudFiles.format", "json") \ ...

  • 980 Views
  • 2 replies
  • 3 kudos
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 3 kudos

Please be sure that the DBR version is 10.5 or highercommit_time and archive_time can be null but discovery_time is set even as NOT NULL in the table definition so it is a bit strange. Please change the DBR version first.

  • 3 kudos
1 More Replies
Labels