nvashisth
New Contributor III

‎08-13-2025 09:40 AM

@timo199 , @boskicl I had similar issue and job was getting stuck at Filtering Files for Query indefinitely. I checked SPARK logs and based on that figured out that we had enabled PHOTON acceleration on our cluster for job and datatype of our columns being used for join have datatype with string collation.

Following was the error message that I got in logs:-

Photon does not fully support the query because:
 collationkey(FAC_PID_FACILITY#5813) is not supported:
The expression cannot be photonized for reason 'UNSUPPORTED_DATATYPE'.	collationkey(OST_RID_OUTSTANDNG#6304) is not supported:
The expression cannot be photonized for reason 'UNSUPPORTED_DATATYPE'
To resolve this, we had turned off PHOTON acceleration setting on cluster and then job succeeded.
 
Hope this might help for some folks.
 
Thanks!
Nitin