GEOMETRY column type breaks all access to table from Spark
Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
09-12-2025 11:43 AM
Hello,
I have a Databricks table with a column using the new GEOMETRY type. When I try to access this table from a Spark workload, I am not able to describe the table or operate on any of its columns. My Spark config is the following, per the Databricks docs:
spark.sql.catalog.spark_catalog org.apache.spark.sql.delta.catalog.DeltaCatalog spark.sql.catalog.my_catalog io.unitycatalog.spark.UCSingleCatalog spark.sql.catalog.my_catalog.uri <Databricks workspace URL>/api/2.1/unity-catalog spark.sql.catalog.my_catalog.token <Databricks PAT>
If I try to access the table from my Spark workload - even to examine its schema or operate on non-geometry columns - I get an error like this:
io.unitycatalog.client.ApiException: com.fasterxml.jackson.databind.exc.ValueInstantiationException: Cannot construct instance of `io.unitycatalog.client.model.ColumnTypeName`, problem: Unexpected value 'GEOMETRY' at [Source: (jdk.internal.net.http.ResponseSubscribers$HttpResponseInputStream); line: 1, column: 8354] (through reference chain: io.unitycatalog.client.model.TableInfo["columns"]->java.util.ArrayList[13]->io.unitycatalog.client.model.ColumnInfo["type_name"]) at io.unitycatalog.client.api.TablesApi.getTableWithHttpInfo(TablesApi.java:273) at io.unitycatalog.client.api.TablesApi.getTable(TablesApi.java:241) at io.unitycatalog.spark.UCProxy.loadTable(UCSingleCatalog.scala:237) at org.apache.spark.sql.connector.catalog.DelegatingCatalogExtension.loadTable(DelegatingCatalogExtension.java:73) at org.apache.spark.sql.delta.catalog.DeltaCatalog.super$loadTable(DeltaCatalog.scala:229) at org.apache.spark.sql.delta.catalog.DeltaCatalog.$anonfun$loadTable$1(DeltaCatalog.scala:229) at org.apache.spark.sql.delta.metering.DeltaLogging.recordFrameProfile(DeltaLogging.scala:171) at org.apache.spark.sql.delta.metering.DeltaLogging.recordFrameProfile$(DeltaLogging.scala:169) at org.apache.spark.sql.delta.catalog.DeltaCatalog.recordFrameProfile(DeltaCatalog.scala:67) at org.apache.spark.sql.delta.catalog.DeltaCatalog.loadTable(DeltaCatalog.scala:228) at io.unitycatalog.spark.UCSingleCatalog.loadTable(UCSingleCatalog.scala:73) at org.apache.spark.sql.connector.catalog.CatalogV2Util$.getTable(CatalogV2Util.scala:363) at org.apache.spark.sql.connector.catalog.CatalogV2Util$.loadTable(CatalogV2Util.scala:337) at org.apache.spark.sql.catalyst.analysis.Analyzer$ResolveRelations$.$anonfun$lookupTableOrView$2(Analyzer.scala:1228) [...]