cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

irfanaziz
by Contributor II
  • 29232 Views
  • 8 replies
  • 8 kudos

Resolved! How to merge small parquet files into a single parquet file?

I have thousands of parquet files having same schema and each has 1 or more records. But reading with spark these files is very very slow. I want to know if there is any solution how to merge the files before reading them with spark? Or is there any ...

  • 29232 Views
  • 8 replies
  • 8 kudos
Latest Reply
Sailaja
New Contributor II
  • 8 kudos

We can combine all these small parquet files into single file using optimize command..Optimize delta_table_name

  • 8 kudos
7 More Replies
Labels