ā12-20-2024 02:32 AM
Hi Team,
I am trying to execute the below API, and it is failing.
API:
curl -v -X POST "https://dbc-xxxxxxxx-xxxx.cloud.databricks.com/api/2.0/unity-catalog/temporary-stage-credentials" -H "Authentication: Bearer xxxxxxxxxxxxxxxx" -d '{"table_id":"external.externalschema.lab_2010","operation_name": "READ"}'
RESPONSE:
{"error_code":"INVALID_PARAMETER_VALUE","message":"GenerateTemporaryStageCredential Missing required field: staging_url","details":[{"@type":"type.googleapis.com/google.rpc.ErrorInfo","reason":"INVALID_FIELD","domain":"unity-catalog.databricks.com","metadata":{"field_name":"staging_url"}},{"@type":"type.googleapis.com/google.rpc.RequestInfo","request_id":"xxxxxx-1a3e-467f-88fc-44c6db8b20e9","serving_data":""}]}
The API is complaining that "staging_url" is required one, but in the API documentation I don't see the "staging_url" is required. Here is the documentation:
Please advise how to proceed with this.
Regards,
Rajesh.
ā12-20-2024 08:43 AM
Your endpoint also seems to be incorrect, as per doc https://docs.databricks.com/api/gcp/workspace/temporarytablecredentials/generatetemporarytablecreden... it has to be /api/2.0/unity-catalog/temporary-table-credentials and you are using /api/2.0/unity-catalog/temporary-stage-credentials
ā12-20-2024 08:47 AM
ā12-20-2024 03:51 AM
Hello, many thanks for the question I have tried the same API call and it has worked as expected, it seems that the id of the table you are passing is incorrect, to get the proper id, please look for the table in the Catalog explorer and then go to the tab details there you will find the table id.
Please make sure that the metastore is enabled with the feature External data access and that the user has EXTERNAL USE SCHEMA permission on the schema where the table was created.
ā12-20-2024 08:04 AM
Hi Walter,
I have checked the required permission, and external data access is enabled. Here are the catalog and metastore details:
CATALOG:
{
"name": "external",
"owner": āXXXXXXXX@xxxxxxx.com",
"storage_root": "s3://xxxxx-xxxxx-test/",
"catalog_type": "MANAGED_CATALOG",
"metastore_id": āxxxxxxx-xxxx-441c-8548-f15054e97666",
"created_at": 1734688660503,
"created_by": āxxxxxxxx@xxxxxx.com",
"updated_at": 1734688660503,
"updated_by": āxxxxxx.xxxxxx@xxxxxx.com",
"storage_location": "s3://xxxxx-xxxxx-xxxxx/__unitystorage/catalogs/xxxxxx-xxxx-4a50-9f8f-84fa64a4be8b",
"isolation_mode": "OPEN",
"accessible_in_current_workspace": true,
"browse_only": false,
"id": āxxxxxxxx-2c9c-4a50-9f8f-xxxxxxxxxxā,
"full_name": "external",
"securable_type": "CATALOG",
"securable_kind": "CATALOG_STANDARD",
"resource_name": "/metastores/xxxxxxxx-334c-441c-8548-xxxxxxx666/catalogs/xxxxxxxx-2c9c-4a50-9f8f-xxxxxxe8b"
},
METASTORE:
{
"metastores": [
{
"name": āxxxxxxxx_aws_us_west_2",
"default_data_access_config_id": āxxxxxxx-4d26-4f21-xxxxx-14008c6b4696",
"storage_root_credential_id": "xxxxxxx-4d26-4f21-xxxxx-14008c6b4696",
"delta_sharing_scope": "INTERNAL",
"owner": "System user",
"privilege_model_version": "1.0",
"region": "us-west-2",
"metastore_id": "xxxxxxx-xxxx-441c-8548-f15054e97666",
"metastore_account_id": āxxxxxxx-a8aa-xxxxx-998a-xxxxxxxxxā,
"created_at": 1734531659103,
"created_by": "System user",
"updated_at": 1734708821326,
"updated_by": āxxxxxxx.xxxxxx@xxxxxx.com",
"storage_root_credential_name": āxxxxx_s3_credentials_xxxxxxxx-xxxxā,
"cloud": "aws",
"global_metastore_id": "aws:us-west-2:xxxxxxxx-xxxx-551c-9558-f25056e98696",
"full_name": "xxxxxxxx_aws_us_west_2",
"securable_type": "METASTORE",
"securable_kind": "METASTORE_STANDARD",
"predictive_optimization_enabled": false,
"external_access_enabled": true
}
]
}
The user has "external use schema" privilege.
Regards,
Rajesh.
ā12-20-2024 08:09 AM
Got it, can you confirm if the table id you are using is correct? This is because in your API you mention this:
'{"table_id":"external.externalschema.lab_2010","operation_name": "READ"}'
On the table id it seems you are using the name of the table which is not correct, you need to go to the details of the table and you will find the table id that should be provided.
ā12-20-2024 08:14 AM
yes, I corrected it
curl -v -X POST "https://dbc-xxxxx-xxxx.cloud.databricks.com/api/2.0/unity-catalog/temporary-stage-credentials" -H "Authentication: Bearer xxxxxxxxxxxxx" -d '{"table_id":"xxxxx-c4b4-4827-a834-xxxxxxx","operation_name": "READ"}'
ā12-20-2024 08:16 AM - edited ā12-20-2024 08:19 AM
You still receiving same exact error or a different one?
If the same error try to run it this way:
curl --location 'https://<workspace_url>/api/2.0/unity-catalog/temporary-table-credentials' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer ***********' \
--data '{
"table_id": "xxxx-c4b4-4827-a834-xxxxxxx",
"operation": "READ"
}'
ā12-20-2024 08:20 AM
I am receiving the same error.
ā12-20-2024 08:28 AM
curl --location 'https://dbc-xxxxxxx-xxxx.cloud.databricks.com/api/2.0/unity-catalog/temporary-stage-credentials' \
> --header 'Content-type: application/json' \
> --header 'Authentication: Bearer xxxxxxxxxxxxxxxxxxx' \
> --data '{"table_id":"xxxx-c4b4-4827-a834-xxxxxxx","operation_name": "READ"}'
{"error_code":"INVALID_PARAMETER_VALUE","message":"GenerateTemporaryStageCredential Missing required field: staging_url","details":[{"@type":"type.googleapis.com/google.rpc.ErrorInfo","reason":"INVALID_FIELD","domain":"unity-catalog.databricks.com","metadata":{"field_name":"staging_url"}},{"@type":"type.googleapis.com/google.rpc.RequestInfo","request_id":"f289990f-a5f7-40e8-8cea-2b7cc25c353e","serving_data":""}]}%
ā12-20-2024 08:29 AM
Can you try this:
curl --location 'https://<workspace_url>/api/2.0/unity-catalog/temporary-table-credentials' \
--header 'Content-Type: application/json' \
--header 'Authorization: Bearer ***********' \
--data '{
"table_id": "xxxx-c4b4-4827-a834-xxxxxxx",
"operation": "READ"
}'
Seems the field is called operation and not operation_name
ā12-20-2024 08:33 AM
same error:
% curl --location 'https://dbc-xxxxxx-xxxx.cloud.databricks.com/api/2.0/unity-catalog/temporary-stage-credentials' \
--header 'Content-type: application/json' \
--header 'Authentication: Bearer xxxxxxxxxxxxxxxxxxxxxxxx' \
--data '{"table_id":"xxxxxx-xxxxx-4827-a834-xxxxxxxxx","operation": "READ"}'
{"error_code":"INVALID_PARAMETER_VALUE","message":"GenerateTemporaryStageCredential Missing required field: staging_url","details":[{"@type":"type.googleapis.com/google.rpc.ErrorInfo","reason":"INVALID_FIELD","domain":"unity-catalog.databricks.com","metadata":{"field_name":"staging_url"}},{"@type":"type.googleapis.com/google.rpc.RequestInfo","request_id":"8a541bb5-ee18-45ea-aedb-0f03eb72dcf0","serving_data":""}]}%
ā12-20-2024 08:43 AM
Your endpoint also seems to be incorrect, as per doc https://docs.databricks.com/api/gcp/workspace/temporarytablecredentials/generatetemporarytablecreden... it has to be /api/2.0/unity-catalog/temporary-table-credentials and you are using /api/2.0/unity-catalog/temporary-stage-credentials
ā12-20-2024 08:47 AM
Wow, it worked. Thank you so much!
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonāt want to miss the chance to attend and share knowledge.
If there isnāt a group near you, start one and help create a community that brings people together.
Request a New Group