Difference between Databricks and local pyspark split.

Merchiv
New Contributor III

I have noticed some inconsistent behavior between calling the 'split' fuction on databricks and on my local installation.

Running it in a databricks notebook gives

spark.sql("SELECT split('abc', ''), size(split('abc',''))").show()

image.pngSo the string is split in 3 parts.

If I run on my local install I get 4 parts.

Locally I'm running on pyspark 3.2.1, on databricks I've tried it with multiple versions but they all give the same result.