@Nithin Tiruveedhi​ Please try as below. Below is an example for word count logic.

val tmpTable1 = sqlContext.sql("select row_number() over (order by count) as rnk,word,count from wordcount")

tmpTable1.registertempTable("wordcount_rownum")

sqlContext.cacheTable("wordcount_rownum")

val df1 = sql("select from wordcount_rownum where rnk <=1000000")

val df2 = sql("select

from wordcount_rowum where rnk between 1000001 and 2000000")

val df3 = sql("select * from wordcount_rowum where rnk between 2000001 and 3000000")

val df4 = sql("select * from wordcount_rowum where rnk between 3000001 and 4000000")

val df5 = sql("select * from wordcount_rowum where rnk between 4000001 and 5000000")

Hope this helps.