Databricks Community

Aamir2028 · a week ago

Hi everyone,

I am using Salesforce trail version. I am getting error which shows "Unexpected error occurred" while connecting it with Databricks free edition, However, I already passed the necessary OAuth with the credentials.

Please advise what action I should take or how to fix this error.

J-Data · a week ago

For what it's worth, I am getting the same error when I try to connect Google Analytics Raw data. I am wondering if this is perhaps a limitation of the Databricks free edition.

data_pulse · a week ago

@Aamir2028

Firstly, welcome to the Databricks Community! Great to see you exploring Salesforce ingestion with Databricks Free Edition.

I have just tested this myself by registering for Salesforce Free Trial account with Enterprise Edition access and a Databricks Free Edition workspace. I was able to authenticate through OAuth and successfully create a Salesforce connection.

To help narrow down the issue, could you please share a few additional details?

At which step does the error occur? Is it during Salesforce OAuth auth, connection creation, or while setting up the ingestion pipeline?
Could you expand the “Unexpected error occurred” message and share the complete error details or a screenshot (with any sensitive information hidden)?
Which Salesforce edition are you using? You mentioned a trial account, is it Enterprise, Developer?
Which authentication method did you select? For example, OAuth User to Machine or Mutual TLS.

Aamir2028 · a week ago

Hi @data_pulse,

Thank you for your quick response. I am using Organization Edition: Starter Edition, please see the attachment. I am using OAuth for the credentials. I get error after selecting catalog and schema to process the further action.

Please advise which version I should use. Please advise how it can be solved.

data_pulse · a week ago

@Aamir2028

Thanks for sharing the additional details and screenshots.

I noticed that your Salesforce organisation is running Starter Edition, whereas my successful connection test used Salesforce Enterprise Edition with Databricks Free Edition.

According to Salesforce documentation, API access is enabled by default for Enterprise, Developer, Unlimited and Performance editions. Starter Edition does not include the same standard API access.

Since your error occurs at the Source step, where Databricks attempts to discover Salesforce objects, my suspicion is that the connector is unable to retrieve the required metadata due to API access restrictions.

Could you check whether your Salesforce user has the API Enabled permission through below?
Method 1 - Check through your Salesforce Profile
1. Log in to Salesforce.
2. Click the Gear icon (top-right) → Setup.
3. In the Quick Find search box, type Users.
4. Navigate to Users → Users.
5. Click your username to open your user details.
6. Find the Profile field and click the profile name.
7. Find API Enabled and check whether it is enabled

Method 2 - Check through Permission Sets
If you cannot find it in the Profile:
1. Go to Setup → Users → Users.
2. Open your user record.
3. Scroll to Permission Set Assignments.
4. Open an assigned permission set.
5. Select System Permissions.
6. Search for API Enabled.

For further validation, I'd suggest trying a Salesforce Developer Edition account, which provides API access for testing: https://developer.salesforce.com/signup

data_pulse · a week ago

@J-Data

Could you as-well post the Error details to see if its connection issue?

J-Data · Wednesday

@data_pulse

The error occurs at the Source step:

After a while (~ 5 minutes) this will result in an Unexpected error:

The authentication works through a BigQuery Service Account which I gave the following roles:
BigQuery Data Viewer
BigQuery Job User
BigQuery Read Session User

data_pulse · Wednesday

@J-Data

Thanks for sharing the screenshot and service account permissions.

The three IAM roles you listed match the docs requirements for Google Analytics Raw Data ingestion through BigQuery.

A few additional checks might help narrow this down:

Could you confirm whether you're using the Google Analytics Raw Data connector (via BigQuery) or the Google BigQuery connector?
Are the BigQuery API, BigQuery Storage API and Cloud Resource Manager API enabled in the relevant Google Cloud project?
Can the service account access and list the source datasets/tables directly in BigQuery?
Does clicking Show details during connection reveal anything beyond the generic message? If possible, please share the detailed error.

The fact that authentication succeeds but Source discovery fails suggests it's worth investigating metadata access and connector-specific prerequisites separately.

I will also try setting up Google cloud environment and try the same connection to Bigquery and will let you know further.

J-Data · Wednesday

Hi @data_pulse

Thanks! I was missing the Cloud Resource Manager API in my enabled API's.
I am now able to proceed with the setup!

Aravind_Reddy · Wednesday

Hi @J-Data

One thing worth distinguishing here is successful OAuth authentication versus successful API access. OAuth can complete successfully, but the connector may still fail when it tries to discover Salesforce objects or retrieve their metadata.

Since the error occurs after selecting the catalog and schema, I'd check whether the Salesforce edition supports the required API access and whether the connected user has the necessary permissions. Testing with a Salesforce Developer Edition account could also help isolate whether the issue is specific to the Starter Edition or the Databricks connection itself.

data_pulse · Wednesday

@J-Data

I tried reproducing this in my Databricks workspace using a Google Cloud BigQuery test project and the Google Analytics Raw Data connector.

I configured the three required Google Cloud APIs (BigQuery, BigQuery Storage and Cloud Resource Manager), created a service account with the documented IAM roles, and authenticated using its JSON key.

My observations:

Unity Catalog connection creation: Successful.
Lakeflow ingestion pipeline creation: Successful.
Reached Step 3 - Source discovery.
Instead of “An unexpected error occurred”, my workspace displayed “No data available. Check your connection, API quota, or if any data is present.”

I haven't configured a GA4 BigQuery export yet, so this is expected to be a possible reason for the empty source list.

Could you confirm whether the Cloud Resource Manager API is enabled in your Google Cloud project? Databricks specifically documents it as a dependency for schema exploration.

data_pulse · Wednesday

@J-Data

Just noticed that you've already identified the missing Cloud Resource Manager API as the issue!

Looks like that was indeed one of the suspects we had in mind, the missing piece of the puzzle was hiding in plain sight! 😄

Could you also mark the reply that resolved your issue as the Accepted Solution? It would be really helpful for others encountering the same problem to find the fix quickly.