Hi @Ashok_Vengala,
This comes up fairly often, so let me break it down. the different behavior you are seeing between older and newer workspaces is expected, and I can help clarify what is happening here.
UNDERSTANDING THE DEPRECATION
The legacy Iceberg REST Catalog API endpoint:
/api/2.1/unity-catalog/iceberg/v1/
has been deprecated in favor of the new endpoint:
/api/2.1/unity-catalog/iceberg-rest/v1/
The legacy endpoint documentation has been officially retired and moved to the archive:
https://docs.databricks.com/aws/en/archive/legacy/external-access-iceberg
ANSWERS TO YOUR SPECIFIC QUESTIONS
1. When was the legacy endpoint disabled for new workspaces?
The deprecation of the legacy endpoint is tied to a control-plane rollout, not a specific workspace version or Databricks Runtime version. Databricks rolls out platform changes progressively across its fleet. Newer workspaces receive the latest control-plane configuration at creation time, which now blocks the legacy endpoint and returns the 400 BadRequestException you are seeing. Older workspaces that were created before this rollout still have the legacy endpoint enabled during a transition period, but they will eventually be migrated as well.
There is no single "workspace version number" or specific date published for this change -- it is part of the continuous platform rollout process.
2. Is this tied to workspace creation date or a feature flag?
It is effectively tied to the control-plane configuration your workspace received at creation time. Think of it as a platform-level rollout rather than something you toggle. Newer workspaces are provisioned with the updated configuration that enforces the new endpoint, while older workspaces are still in a grace period. Both old and new workspaces support the new /iceberg-rest/ endpoint, so migrating now ensures you are ready regardless of when the legacy endpoint is fully retired on older workspaces.
3. Does the deprecation depend on the Databricks Runtime version?
No. The Iceberg REST Catalog API is a control-plane service, not a cluster-level feature. The "Databricks Runtime 16.4 LTS and above" requirement mentioned in the documentation refers to when the new Iceberg REST Catalog endpoint feature became available in Public Preview -- it does not mean the API depends on the DBR version running on your clusters. The API endpoint is served by the Databricks control plane independently of any cluster or Databricks Runtime version.
4. Where can I see which DBR version is associated with a workspace for API access?
There is no DBR version associated with your workspace for API access purposes. The Databricks Runtime version only applies to compute clusters (all-purpose clusters, job clusters, SQL warehouses). The Iceberg REST Catalog API is a workspace-level control-plane service that operates independently of any cluster runtime. What matters for the API is the workspace's control-plane version, which Databricks manages automatically and is not directly exposed to users.
WHAT YOU SHOULD DO
Since the legacy endpoint is deprecated and will eventually be disabled on all workspaces, the recommended action is to migrate to the new endpoint now. The migration is straightforward -- update the catalog URI in your client configurations:
Old:
<workspace-url>/api/2.1/unity-catalog/iceberg/v1/
New:
<workspace-url>/api/2.1/unity-catalog/iceberg-rest/v1/
For example, in an Apache Spark configuration:
spark.sql.catalog.<name>.uri = <workspace-url>/api/2.1/unity-catalog/iceberg-rest
Important: Make sure your workspace URL includes the workspace ID. Without it, API requests may return a 303 redirect to a login page instead of the expected response.
BENEFITS OF THE NEW ENDPOINT
The new endpoint is a significant upgrade:
1. Read AND write support -- The legacy endpoint was read-only. The new one supports both read and write operations for managed Iceberg tables.
2. Higher throughput -- The legacy endpoint had a 5 QPS rate limit. The new endpoint supports substantially higher throughput.
3. Broader table type support:
- Managed Iceberg tables (read + write)
- Foreign Iceberg tables (read-only)
- Managed Delta tables with Iceberg reads enabled via UniForm (read-only)
- External Delta tables with Iceberg reads enabled via UniForm (read-only)
4. Credential vending -- Provides temporary credentials to external engines for direct storage access.
The new endpoint is currently in Public Preview.
KEY REQUIREMENTS
Before using the new endpoint, confirm:
- Unity Catalog is enabled on your workspace
- External data access is enabled on your metastore
- The EXTERNAL USE SCHEMA privilege is granted to the principal accessing the data
- Your workspace URL includes the workspace ID
RELEVANT DOCUMENTATION
- Current Iceberg external access guide: https://docs.databricks.com/aws/en/external-access/iceberg
- Legacy (archived) Iceberg endpoint docs: https://docs.databricks.com/aws/en/archive/legacy/external-access-iceberg
- Credential vending: https://docs.databricks.com/aws/en/external-access/credential-vending
- Managed Iceberg tables: https://docs.databricks.com/aws/en/tables/managed
- Iceberg overview: https://docs.databricks.com/iceberg/
Hope this helps clarify things. The key takeaway is: migrate to /api/2.1/unity-catalog/iceberg-rest/v1/ on all your workspaces now, and you will get a consistent experience across old and new workspaces along with better performance and read-write capabilities.
* This reply used an agent system I built to research and draft this response based on the wide set of documentation I have available and previous memory. I personally review the draft for any obvious issues and for monitoring system reliability and update it when I detect any drift, but there is still a small chance that something is inaccurate, especially if you are experimenting with brand new features.
