topic Re: date_format 'LLLL' returns '1' in Data Engineering

date_format 'LLLL' returns '1'

Kibour — Wed, 13 Mar 2024 13:54:44 GMT

Hi all,

In my notebook, when I run my cell with following code

%sql

select date_format(date '1970-01-01', "LLL");

I get '1', while I expect 'Jan' according to the doc
https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html

I would also expect the following to return 'January', but I get '1' instead

%sql

select to_csv(named_struct('date', date '1970-01-01'), map('dateFormat', 'LLLL', 'locale', 'EN'));

What am I missing?

Thanks in advance for your insights

Re: date_format 'LLLL' returns '1'

Kibour — Fri, 15 Mar 2024 15:30:47 GMT

Hi @Retired_mod ,

Thanks for your detailed answer. If I read you correctly, this means the spark docuementation is incorrect. I copy-pasted my queries direclty from the documentation examples, but can't reproduce them.

My cluster is runnig with Apache Spark 3.5.0, Scala 2.12.

Thanks and kind regards

Re: date_format 'LLLL' returns '1'

Kibour — Tue, 09 Apr 2024 09:38:25 GMT

Hi @Retired_mod ,

Turns out it was actually a Java 8 bug:

IllegalArgumentException: Java 8 has a bug to support stand-alone form (3 or more 'L' or 'q' in the pattern string). Please use 'M' or 'Q' instead, or upgrade your Java version. For more details, please read https://bugs.openjdk.java.net/browse/JDK-8114833

Upgarding to Java 11 fixed the issue.

Kind regards