cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

alesventus
by Contributor
  • 2566 Views
  • 1 replies
  • 2 kudos

Pyspark Merge parquet and delta file

Is it possible to use merge command when source file is parquet and destination file is delta? Or both files must delta files? Currently, I'm using this code and I transform parquet into delta and it works. But I want to avoid of this tranformation.T...

  • 2566 Views
  • 1 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Ales ventus​ We haven't heard from you since the last response from @Kaniz Fatma​ , and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to others...

  • 2 kudos
User16826994223
by Honored Contributor III
  • 1954 Views
  • 1 replies
  • 0 kudos

Delta Lake MERGE INTO statement error

I'm trying to run Delta Lake MergeMERGE INTO source USING updates ON source.d = updates.sessionId WHEN MATCHED THEN UPDATE * WHEN NOT MATCHED THEN INSERT *I'm getting an SQL errorParseException: mismatched input 'MERGE' expecting {'(', 'SELECT', 'FR...

  • 1954 Views
  • 1 replies
  • 0 kudos
Latest Reply
User16826994223
Honored Contributor III
  • 0 kudos

The merge SQL support is added in Delta Lake 0.7.0. You also need to upgrade your Apache Spark to 3.0.0 and enable the integration with Apache Spark DataSourceV2 and C

  • 0 kudos
brickster_2018
by Databricks Employee
  • 1638 Views
  • 1 replies
  • 0 kudos
  • 1638 Views
  • 1 replies
  • 0 kudos
Latest Reply
brickster_2018
Databricks Employee
  • 0 kudos

The impact will be only on the files touched by the MERGE operation. The newly created files will not be optimized and data co-locality is not ensured. However, the files which are not touched by the MERGE operation will continue to show the improvem...

  • 0 kudos
Labels