Databricks integration with DocumentDB

[jefft]

Hi Everyone,

I was wondering if anyone here has any experience or tips reading data from AWS DocumentDB. I am working on this using the MongoDB connector. For DocumentDB we also need to work with the required creds issued as a .pem file by AWS. Thanks for any pointers anyone would be willing to share if they have gotten this working.

[Kaniz]

Hi @Jeffrey Tennenbaum​, Databricks Runtime provides bindings to popular data sources and formats to make importing and exporting data from the lakehouse simple.

This article provides information to help you identify formats and integrations that have built-in support.

You can also discover ways to extend Databricks to interact with even more systems.

Databricks provides a number of optimizations for data loading and ingestion.

Databricks also supports query federation for both SQL and DataFrame users. See What is query federation?

If you have not read or write data with Databricks before, consider reviewing the DataFrames tutorial for Python or Scala. Even for users familiar with Apache Spark™, this tutorial might address new challenges associated with accessing data in the cloud.

Partner Connect provides optimized, easy-to-configure integrations to many enterprise solutions. See What is Databricks Partner Connect?

[Kaniz]

Hi @Jeffrey Tennenbaum​ ​, We haven’t heard from you since the last response from me​ ​, and I was checking back to see if my suggestions helped you.

Or else, If you have any solution, please share it with the community, as it can be helpful to others.

Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.

[jefft]

Hi @Kaniz Fatma​ ,

Thank you so much for your response. Your suggestions were helpful. As per the AWS documentation, DocumentDB is MongoDB compatible. "With Amazon DocumentDB, you can run the same application code and use the same drivers and tools that you use with MongoDB." With that in mind, we are looking to connect using the MongoDB drivers. We have not been able to get the connection working between Databricks, and DocumentDB yet, hence my initial question. Thanks again.