There is no offset support yet. Here are a few possible workarounds
- If you data is all in one partition ( rarely the case ๐ ) , you could create a column with monotonically_increasing_id and apply filter conditions. if there are multiple partitions, monotonically_increasing_id won't be consecutive
- Use except ( in your case sql equivalent of code below) . This however would be an expensive operation
df1 = df.limit(10);
df2 = df.except(df1);
df2.limit(20);