Databricks Community

I just tried to setup a row filter via the following two sql snippets:create function if not exists foo.my_test.row_filter (batch_id bigint) return TRUE;alter table foo.my_test.some_table set row filter foo.my_test.row_filter on (batch_id); This resu...

I just opened another issue: https://issues.apache.org/jira/browse/SPARK-46959It corrupts data even when read with mode="FAILFAST", i consider it critical, because basic stuff like this  should just work!

I already reported that as a Bug to the official Spark bug tracker: https://issues.apache.org/jira/browse/SPARK-46876A short summary: When reading a tab separated file, that has lines that only contain of tabs, then this line will not show up in the ...