Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
05-13-2025 10:05 AM
Hello,
I am trying to migrate some spark reads away from JDBC into the federated queries based in unity catalog.
Here is an example of the spark read command that I want to migrate:
spark.read.format("jdbc").option("driver", "org.postgresql.Driver").option("url", url).option("user", user).option("password", password).option("query", query).option("preferTimestampNTZ", True).load()That "preferTimestampNTZ" option is really important because I have many table columns in the source that timestamps without timezone and are not in UTC.
Problem is that I cannot find in the documentation on how to set up the catalog to leverage this option.
The closest that I can find is at the bottom of this link here: Run federated queries on PostgreSQL | Databricks Documentation but that doesn't explain how that option can be applied.
Here is an example of how I want the future queries to look like:
spark.table("postgres_catalog.postgres_database.postgres_table")
Does anyone know?
Thank You
