cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 
Help
Machine Learning
Dive into the world of machine learning on the Databricks platform. Explore discussions on algorithms, model training, deployment, and more. Connect with ML enthusiasts and experts.
cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
alesventus
by • Contributor
  • 3036 Views
  • 1 replies
  • 2 kudos

Pyspark Merge parquet and delta file

Is it possible to use merge command when source file is parquet and destination file is delta? Or both files must delta files? Currently, I'm using this code and I transform parquet into delta and it works. But I want to avoid of this tranformation.T...

Machine Learning
Reply
  • 3036 Views
  • 1 replies
  • 2 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Ales ventus​ We haven't heard from you since the last response from @Kaniz Fatma​ , and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to others...

  • 2 kudos
User16826994223
by • Honored Contributor III
  • 2180 Views
  • 1 replies
  • 0 kudos

Delta Lake MERGE INTO statement error

I'm trying to run Delta Lake MergeMERGE INTO source USING updates ON source.d = updates.sessionId WHEN MATCHED THEN UPDATE * WHEN NOT MATCHED THEN INSERT *I'm getting an SQL errorParseException: mismatched input 'MERGE' expecting {'(', 'SELECT', 'FR...

Machine Learning
Reply
  • 2180 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
User16826994223
Honored Contributor III
  • 0 kudos

The merge SQL support is added in Delta Lake 0.7.0. You also need to upgrade your Apache Spark to 3.0.0 and enable the integration with Apache Spark DataSourceV2 and C

  • 0 kudos
brickster_2018
by • Databricks Employee
  • 1908 Views
  • 1 replies
  • 0 kudos

Resolved! After applying a MERGE in a previously Z-ORDERED table, Will the table loses the entire Z-ORDER optimization or only on the files touched by the MERGE?

Machine Learning
Reply
  • 1908 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
brickster_2018
Databricks Employee
  • 0 kudos

The impact will be only on the files touched by the MERGE operation. The newly created files will not be optimized and data co-locality is not ensured. However, the files which are not touched by the MERGE operation will continue to show the improvem...

  • 0 kudos
Labels
Top