If you’re using R I highly recommend the sparklyr package from RStudio. Many of the pyspark functions have the same name, for example, “spark.read.table()” is “spark_read_table” in sparklyr. More info here: https://spark.rstudio.com/packages/sparklyr/latest/reference/spark_read_table.html

View solution in original post