Databricks Community

Hi Nithin, You can use the DataFrame's randomSplit function. For example: val df = sc.parallelize(1 to 10000).toDF("value") val splitDF = df.randomSplit(Array(1,1,1,1,1)) val (df1,df2,df3,df4,df5) = (splitDF(0),splitDF(1),splitDF(2),splitDF(3),spl...

Could you possible supply some Python code that creates a small DataFrame that demonstrates this behavior?

You could try to code your own version of this. It does not seem like this functionality is built into the Spark DataFrames. You may need to use the Window class in order to accomplish this. Here is a blog post with some details: https://databricks....