UC Delta API: managed-table READ_WRITE credentials rejected despite explicit User-Agent

[ablanchard]

Hi everyone,

I’m developing HarborSQL, an open-source SQL engine using Apache DataFusion and delta-rs. I’m working on managed Delta table writes through the UC Delta API with catalog commits.

Using the same workspace, managed table, and user identity:

- Table metadata GET succeeds (200).
- Credentials GET with operation=READ succeeds (200).
- Credentials GET with operation=READ_WRITE fails (400).

This happens with both PAT and browser-based OAuth U2M authentication.

The failing request is:

GET /api/2.1/unity-catalog/delta/v1/catalogs/{catalog}/schemas/{schema}/tables/{table}/credentials?operation=READ_WRITE
User-Agent: HarborSQL_HarborSQL/0.1.9
Accept: application/json
Content-Type: application/json

The response says:

The provided User-Agent 'HarborSQL_HarborSQL/0.1.9' is insufficient.

It also directs connector developers to contact Databricks support, but I don’t have access to support.

The header follows the documented partner User-Agent format. Using harborsql/0.1.9 produces the same error. The failure occurs during credential vending, before any data write or commit request.

Does managed-table write access require connector registration, allowlisting, or additional preview enablement? Are there client-identification requirements beyond the documented User-Agent format?

If onboarding is required for independent open-source engines implementing catalog commits, could someone point me to the appropriate process or team?

Thanks!

[GabFernandes]

Hi @ablanchard .

The behavior you're seeing is expected for the Delta protocol-level credential endpoint (/api/2.1/unity-catalog/delta/v1/.../credentials). A few things to note:

1. Managed Delta table writes from external Delta clients are in Public Preview (docs)During this preview, the Delta protocol API validates the User-Agent header for READ_WRITE operations to gate access to recognized/registered connectors. This is why READ succeeds (it's GA) while READ_WRITE fails.
2. Consider using the general credential vending API instead. The documented public endpoint is POST /api/2.1/unity-catalog/temporary-table-credentials with a JSON body containing table_id(the table's UUID) and operation: "READ_WRITE". This is the officially documented path for external engines and may not enforce the same User-Agent gating. You can get the table UUID via the ListTables API with include_manifest_capabilities enabled — look for tables marked HAS_DIRECT_EXTERNAL_ENGINE_WRITE_SUPPORT.
3. Prerequisites: the metastore must have external_access_enabled = true, and the principal needs EXTERNAL_USE_SCHEMA on the parent schema. For catalog commits specifically, the table property delta.feature.catalogManaged must be set to 'supported'.
4. For the Delta protocol path specifically, connector registration/onboarding with Databricks is likely required during the Public Preview phase. Filing a feature request via the Databricks Ideas portal or opening a GitHub issue on the unitycatalog repo would be the best way to request formal onboarding for HarborSQL.

   If my answer was helpful, please consider marking it as accepted solution!