Thank You i just found a solution, and i have mentioned it in my question to, while reading my file all i had to do was add this,

option("maxRowsInMemory", 1000).

View solution in original post

found a way

Hi @sarvesh singh​ ,

Could you mark the "best" solution to your question please? it will help in case other community members have the se issue in the future.

Thank you

done