by
sanjay
• Valued Contributor II
- 13509 Views
- 8 replies
- 0 kudos
Hi,I have pipeline running. I have updated one file in delta table which is already processed. Now I am getting errorcom.databricks.sql.transaction.tahoe.DeltaUnsupportedOperationException: Detected a data update. This is currently not supported. If ...
- 13509 Views
- 8 replies
- 0 kudos
Latest Reply
Hi Sanjay, You can try adding .option("overwriteSchema", "true")
7 More Replies
by
sanjay
• Valued Contributor II
- 3062 Views
- 4 replies
- 1 kudos
Hi,I am running autoloader which is running continuously and checks for new file every 1 minute. I need to store when file was received/processed but its giving me date when autoloader started. Here is my code.df = (spark .readStream .format("clo...
- 3062 Views
- 4 replies
- 1 kudos
Latest Reply
Hi @Sanjay Jain , You can use the File Metadata column functionality to collect that information.Ref doc:- https://docs.databricks.com/ingestion/file-metadata-column.html
3 More Replies