cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

alesventus
by New Contributor III
  • 1529 Views
  • 2 replies
  • 3 kudos

Pyspark Merge parquet and delta file

Is it possible to use merge command when source file is parquet and destination file is delta? Or both files must delta files? Currently, I'm using this code and I transform parquet into delta and it works. But I want to avoid of this tranformation.T...

  • 1529 Views
  • 2 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Ales ventus​ We haven't heard from you since the last response from @Kaniz Fatma​ , and I was checking back to see if her suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to others...

  • 3 kudos
1 More Replies
Kaniz
by Community Manager
  • 1693 Views
  • 3 replies
  • 0 kudos
  • 1693 Views
  • 3 replies
  • 0 kudos
Latest Reply
armen23
New Contributor II
  • 0 kudos

Code To Merge 2 dict in Python is:def Merge(dict1, dict2):    return(dict2.update(dict1))     # Driver codedict1 = {'a': 10, 'b': 8}dict2 = {'d': 6, 'c': 4}# This return Noneprint(Merge(dict1, dict2))# changes made in dict2print(dict2)For More Visit:...

  • 0 kudos
2 More Replies
User16826994223
by Honored Contributor III
  • 1523 Views
  • 1 replies
  • 0 kudos

Delta Lake MERGE INTO statement error

I'm trying to run Delta Lake MergeMERGE INTO source USING updates ON source.d = updates.sessionId WHEN MATCHED THEN UPDATE * WHEN NOT MATCHED THEN INSERT *I'm getting an SQL errorParseException: mismatched input 'MERGE' expecting {'(', 'SELECT', 'FR...

  • 1523 Views
  • 1 replies
  • 0 kudos
Latest Reply
User16826994223
Honored Contributor III
  • 0 kudos

The merge SQL support is added in Delta Lake 0.7.0. You also need to upgrade your Apache Spark to 3.0.0 and enable the integration with Apache Spark DataSourceV2 and C

  • 0 kudos
User16869510359
by Esteemed Contributor
  • 1116 Views
  • 1 replies
  • 0 kudos
  • 1116 Views
  • 1 replies
  • 0 kudos
Latest Reply
User16869510359
Esteemed Contributor
  • 0 kudos

The impact will be only on the files touched by the MERGE operation. The newly created files will not be optimized and data co-locality is not ensured. However, the files which are not touched by the MERGE operation will continue to show the improvem...

  • 0 kudos
Labels