Delta Sharing from Databricks to SAP BDC fails with invalid_client error

[4Twannie]

Context

We are in the process of extracting data between SAP BDC Datasphere and Databricks (Brownfield Implementation).

• SAP Datasphere is hosted in AWS (eu10)
• Databricks is hosted in Azure (West Europe)
• The BDC Connect System is located in the same region as SAP Datasphere (eu10).

So far, we’ve successfully:

• Set up the BDC Connect connection from SAP Datasphere to Databricks.
• Shared data from SAP to Databricks, which is now available in our catalog.
• Read data from these delta shares in Databricks notebooks.

✅Extracting and reading SAP data in Databricks works fine (even though it is slow, which would be a question for another discussion).

The Problem

When trying to exchange data from Databricks back to SAP, things break down. We followed the official Databricks Delta Sharing Instructions, and everything worked until the last part of Step 6:

"Use Delta Sharing to share and receive data between Databricks and SAP BDC"

Specifically, the link: "Grant SAP Business Data Cloud (BDC) recipients access to Delta Sharing data shares."

Here's what we did:

• Created the share.
• Shared it with the BDC Connect connection we had set up earlier.
• Used notebook to try to describe the share in Core Schema Notation so SAP BDC users could read it.

Notebook snippet (with variables filled in):

bdc_connect_client = BdcConnectClient(DatabricksClient(dbutils, "bdc-connect-test"))

share_name = "test_share"

open_resource_discovery_information = {
    "@openResourceDiscoveryV1": {
        "title": "Title to Share",
        "shortDescription": "Share Short Description",
        "description": "Share Description"
    }
}

catalog_response = bdc_connect_client.create_or_update_share(
    share_name,
    open_resource_discovery_information
)

Error returned:

ValueError: Error when trying to obtain 'access_token' from JSON response: {'error_code': 'INTERNAL_ERROR', 'message': 'Failed to retrieve partner token: INTERNAL: INTERNAL_ERROR: Unable to get user object using principal context with error invalid_client. Please try again or contact support if the issue persists.', 'details': [{'@type': 'type.googleapis.com/google.rpc.RequestInfo', 'request_id': 'c81582e9-e3f0-9d59-8ee3-7c3bbb463b92', 'serving_data': ''}]}

What we've tried so far

• Changed the compute setup: tested both serverless (across multiple environments) and all-purpose compute using Databricks Runtime 17.3 LTS (Apache Spark 4.0.0, Scala 2.13). Side note: across these requests, we also tested multiple versions of the BDC Connect client library (from 1.1.4 down to 1.1.1) on the different computes.
• Updated the description of the share information.
• Re-ran the notebook with different values.

None of these attempts resolved the issue.

Question

Has anyone successfully set up Delta Sharing from Databricks back into SAP BDC Datasphere?

• Is there a specific configuration required for the Core Schema Notation step?
  
• Could this be an authentication/BDC Connect principal context issue?
• Any known workarounds for the invalid_client error when retrieving the token?

Any guidance, experiences, or best practices would be greatly appreciated!

[Abeshek]

This is a common challenge in enterprise SAP Datasphere and Databricks integrations, particularly in brownfield, cross-cloud setups. We’ve seen multiple cases where sharing between SAP and Databricks works as expected, while the reverse path introduces additional complexity related to identity, principal context, and partner trust that is not always evident from the documentation.

In similar SAP BDC–Databricks implementations, the issue has usually been less about Delta Sharing mechanics and more about how authentication and BDC Connect context are established across clouds and runtimes. Once that layer is aligned, the downstream steps tend to fall into place.

If helpful, we’ve supported teams through comparable bidirectional sharing scenarios and can share what has worked in practice.