cancel
Showing results for 
Search instead for 
Did you mean: 
Community Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Expose delta table data to Salesforce - odata?

Ruby8376
Valued Contributor

HI Looking for suggestiongs to stream on demand data from databricks delta tables to salesforce.

Is odata a good option? 

7 REPLIES 7

Ruby8376
Valued Contributor

@-werners- @WernerS  can u help?

-werners-
Esteemed Contributor III

So do you want Databricks to push data into Salesforce or do you want Salesforce to pull data?
If you want to push, you could write microbatches to salesforce (if possible of course).
For pull, you might wanna use the SQL api of databricks sql, OR use some kind of pub/sub tool like kafka etc where all changes are published, which can be fetched later on.

So perhaps some more info on 'Salesforce'.  Is it salesforce cloud or the database?

Ruby8376
Valued Contributor

@-werners- we don't want to persist data on SF cloud but just show it on the ui. It will be pulled on demand from salesforceside using external objects. is odata good option?

-werners-
Esteemed Contributor III

I would not look at OData, as Databricks itself has no out-of-the-box OData integration. So this means you would have to do this yourself.
Instead I would look at the databricks-sql sql api. So the UI you are talking about is doing a REST call to the query api and databricks sql sends the data.
Something like that?

Ruby8376
Valued Contributor

even for this case, there would be some custom development. Salesforce doesn't natively support sending SQL queries directly to external databases or APIs. therefore, either we would need middleware between salesforce and databricks or develope custom apex code on salesforce(to authenticate, request, process repsonse, etc).

Ruby8376
Valued Contributor

Hi @werners

So, we have decided on this flow: Salesforce connect <->APIM <->Webapp(Odata)<->databricks sql API <- Delta tables

 

2 questions here:

1. can you share link/documentation on how we can integrate databricks <-delta tables? do we need serverless sql warehouse for sure or can we just use a databricks sql endpoint for querying??

2. For authentication, we have chosen Oauth machineto machine authentication as per microsoft recommendation. in this process do we use external table to share the same delta table? OAuth machine-to-machine (M2M) authentication | Databricks on AWS

 

@-werners-  Please help

 

-werners-
Esteemed Contributor III

I see, is there a possibility in SF to define an external location/datasource?
Just guessing here, as these type of packages are really good in isolating data, not integrating it.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!