API to write into Databricks tables
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-15-2022 05:40 AM
Hello Experts,
Is there any API in databricks that allows to write the data in the Databricks tables. I would like to send small size Logs information to Databricks tables from other service. What are my options?
Thank you very much.
- Labels:
-
Databricks Tables
-
Tables
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-15-2022 06:55 AM
Hi @Viren Devi ,
I guess one option of writing data into the Databricks tables or Delta Tables to be specific is to use for example Delta Tables writer: https://delta-io.github.io/delta-rs/python/usage.html#writing-delta-tables
I am not sure how this will work with the Databricks Unity Catalog managing access to the data, TBH.
Depends on your requirements. You can write the data into the landing location, then load it into Delta Tables every hour/day.
Thanks,
Pat.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-18-2022 12:20 AM
thank you Pat. I will check this and test. Appreciate the support.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
12-19-2022 08:23 AM
Hello,
I think I was not exactly clear in my requirement. I would like to write into Delta tables from outside the databricks i.e other cloud services. I believe write API works when we are writing the data when in the databricks instance itself?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-17-2022 07:48 AM
and what about use the jdbc/odbc driver, either programatically or using a tool like dbeaver?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
11-18-2022 12:24 AM
we do have a custom driver to pull the data out but we would need to change it for writing it in Databricks. Will try HTTP API first and if unsuccessful go with driver approach. Thanks for the nice suggestion.
![](/skins/images/B38AF44D4BD6CE643D2A527BE673CCF6/responsive_peak/images/icon_anonymous_message.png)
![](/skins/images/B38AF44D4BD6CE643D2A527BE673CCF6/responsive_peak/images/icon_anonymous_message.png)