cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Data_Analytics1
by Contributor III
  • 752 Views
  • 1 replies
  • 2 kudos

Resolved! File not found error. Does OPTIMIZE deletes initial versions of the delta table?

df = (spark.readStream.format("delta")\              .option("readChangeFeed", "true")\              .option("startingVersion", 1)\              .table("CatalogName.SchemaName.TableName")             )display(df)A file referenced in the transaction l...

  • 752 Views
  • 1 replies
  • 2 kudos
Latest Reply
swethaNandan
New Contributor III
  • 2 kudos

Had you run vacuum on the table? Vacuum can clean up data files marked for removal and are older than retention period.Optimize compacts files and marks the small files for removal, but does not physically remove the data files

  • 2 kudos
Labels