cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
twotwoiscute
New Contributor
since ‎08-27-2021
‎06-26-2023

User Stats

  • 2 Posts
  • 0 Solutions
  • 0 Kudos given
  • 0 Kudos received

User Activity

Difference between running pyspark code by using commend python3 and pyspark

07-16-2021
I am confused by what's difference between running code using command python3 CODENAME.py and launch it by commend pyspark and start working on the code. When I run the code : spark = SparkSession.builder.config("spark.driver.memory", "16").appName(...

PySpark pandas_udf slower than single thread

07-14-2021
I used @pandas_udf write a function for speeding up the process(parsing xml file ) and then compare it's speed with single thread , Surprisingly , Using @pandas_udf is two times slower than single-thread code. And the number of xml files I need to p...