UNSUPPORTED_TIME_TYPE despite 18.1 runtime?

[js5]

Hello,

I have tried using TimeType data type which is supported since Spark 4.1:

https://spark.apache.org/docs/latest/sql-ref-datatypes.html

I am unfortunately still getting UNSUPPORTED_TIME_TYPE error when trying to run display() on a pandas dataframe containing time objects. Is this expected? If so, are there any plans for incorporating this new data type in databricks? Thanks!

[Ashwin_DSA · Accepted solution]

Hi @js5,

This is expected today on Databricks. You can check this out for reference.

Spark 4.1 introduces a standard TIME type (TimeType) in the SQL type system, and Databricks runtimes based on Spark 4.x already expose it at the engine level (for example, via functions like current_time). However, Databricks still treats TIME as unsupported in several higher‑level components, including the path that display() uses when converting a pandas DataFrame to a Spark DataFrame. That’s why you see [UNSUPPORTED_TIME_TYPE] The data type TIME is not supported. even though the Spark docs show TimeType.

Full platform‑level support for TIME is being rolled out progressively... You can see the underlying Spark work landing in the Databricks Runtime 17.x release notes (search for “TimeType” and “TIME data type” there). 

 https://docs.databricks.com/en/archive/runtime-release-notes/17.0
 https://docs.databricks.com/en/archive/runtime-release-notes/17.1
 https://docs.databricks.com/en/archive/runtime-release-notes/17.2

Until that’s complete, a practical workaround is to convert pure time‑of‑day columns to STRING in pandas before calling display(), 

If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.