cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Loading parquet files with earlier timestamp looking for newer files

mihirkum123
Visitor

I have a setup where i am replicating Delta live tables parquet, and checkpoint files using azure RAGZRS to peer region for disaster recovery. When i load the the replicated files in peer region using delta format, i get an error that _delta_log/000000000000000_x.json not found. I am loading as of 48 hours in past so i know that all the files before that is replicated. Moreover, x is the of most recent JSON file number which is supposed to be replicated from primary. I tried using timestamp as of,  and version as of and both result in same error. Error disappears as soon as the JSON file is replicated from primary. Is this expected behavior? I would imagine that if i am loading from past, i will not see this error. 

0 REPLIES 0

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group