Spark pushdown filter not being respected on dbfs

[raman]

I have a parquet files with a column g1 with schema

StructField(g1,IntegerType,true)

Now I have a query with filter on g1.

What's weird in the SQL viewer is that spark is loading all the rows from that file.

Even though in the physical plan I can see the "pushedFilter" condition being set.

This is created as a delta table on dbfs.

Any pointer on this would be helpful.

Thanks

[Ajay-Pandey]

Hi @Raman Gupta​ could you please share your code and physical plan for the same

[raman]

Thanks @Ajay Pandey​ pls find attached the physical plan.

Query: Select identityMap, segmentMembership, _repo, workEmail, person, homePhone, workPhone, workAddress, personalEmail, homeAddress from final_segment_index_table_v2 where (g1 >= 128 AND g1 < 256).

Thanks