Genrated partition column not being used by optimizer

AlexDavies
Contributor

We have created a table using the new generated column feature (https://docs.microsoft.com/en-us/azure/databricks/delta/delta-batch#deltausegeneratedcolumns)

CREATE TABLE ingest.MyEvent(
 data  binary,
 topic string,
 timestamp timestamp,
 date date GENERATED ALWAYS AS (CAST(timestamp AS DATE))
)
USING DELTA
PARTITIONED BY (topic,date)
LOCATION '/mnt/data/myevent'

When running

SELECT * FROM WHERE timestamp > '2021-06-01T12:34'

We were expecting it to push down a filter "date >= '2021-06-01'" so that it can make use of the partitioned column without having to explicit query it. However it does not push down a filter on date, so doesn't benefit from partition filtering

Is it expected that generated columns would be used to optimize queries? Is it possible to make generated columns work the way i'm expecting or am I expecting too much?