if file is renamed with file_name.sv.gz (lower case extension) is working fine, if file_name.sv.GZ (upper case extension) the data is read as corrupted, means it simply reading compressed file as is.
Recently I restarted look at a solution for this issue, I found out we can add few exception for allowing "GZ" in hadoop library as GzipCodec is invoked from there.
I assume it should worth handling such thing, as filename or extension should not be a constraint to process data.As we know it's a gzip file and we can pass the paramter to read it as gzip. Thanks a lot for your responses @Lakshay .
Yup, Spark does infer it from filename, I have been through spark code in Github.Article is also refering to the internal code from Spark library. I assume we can add an exception to handle .GZ file as gzip by tweaking spark libraries.
I assume, if a .gz file is renamed as .GZ purposefully then we need to consider it as valid file format as gzip file. Cause that .GZ file still consist compressed data which is still valid.
