cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

whatthespark
by New Contributor II
  • 2428 Views
  • 4 replies
  • 1 kudos

Inconsistent duplicated row with Spark (Databricks on MS Azure)

I'm having a weird behavior with Apache Spark, which I run in a Python Notebook on Azure Databricks. I have a dataframe with some data, with 2 columns of interest: name and ftimeI found that I sometime have duplicated values, sometime not, depending ...

  • 2428 Views
  • 4 replies
  • 1 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 1 kudos

I would like to see how you create the df dataframe.In pyspark you can get weird results if you do not clear state, or when you reuse dataframe names.

  • 1 kudos
3 More Replies
Labels