szymon_dybczak
Esteemed Contributor III

Hi @Malthe ,

read_files accepts format-specific options, but the Avro option table has an important scope column. For mode, that scope is only from_avro, not Avro file reads. Spark documents it as:

"mode … allows you to specify parse mode for function from_avro"

As a workaround,  if your blobs are raw Avro-encoded records/messages, rather than standard Avro Object Container Files, read them as binary and use from_avro, where mode = PERMISSIVE really is supported:

SELECT
from_avro(
content,
'<avro schema>',
map('mode', 'PERMISSIVE')
) AS decoded
FROM read_files(
's3://bucket/path',
format => 'binaryFile'
);

View solution in original post