topic Re: Is there any way we can use usermetadataAsOf option in time travelling query or can we modify the timestamps of delta lake that seems to be immutable? in Data Engineering

Is there any way we can use usermetadataAsOf option in time travelling query or can we modify the timestamps of delta lake that seems to be immutable?

Priyanka48 — Wed, 31 Aug 2022 07:31:29 GMT

We are using delta lake time travelling capability in our current project. We can use select * from timestamp/versionAsOF query. However ,there might be some change in our approach and we might need to recreate the delta lake while persisting the time travelling capability. We have checked the option of using usermetadata to persist original timestamps while recreating delta lake. however, it seems we can not use usermetdataAsOf option directly in time travelling query. Can we get the feature to modify the timestamps in delta table history or can we get an option to use usermetadataAsOf option ?

Re: Is there any way we can use usermetadataAsOf option in time travelling query or can we modify the timestamps of delta lake that seems to be immutable?

Priyanka48 — Mon, 12 Sep 2022 11:13:08 GMT

yes. So I asked if we can get usermetadata as of feature or can we edit the history of delta tables for e.g timestamp columns?

Re: Is there any way we can use usermetadataAsOf option in time travelling query or can we modify the timestamps of delta lake that seems to be immutable?

Debayan — Fri, 02 Sep 2022 21:58:00 GMT

Hi Priyanka, Thanks for reaching out to community.databricks.com.

As of now, "As of" query only has two parameters timestamp and version.

Please refer: https://docs.delta.io/latest/delta-batch.html#sql-as-of-syntax

Please let us know in case if you have any further queries on the same.