cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Delta RUST API (not REST )

prasadvaze
Valued Contributor II

@dennylee

Delta RUST API seems a good option to query delta table without spinning up spark cluster so I am trying out this - https://databricks.com/blog/2020/12/22/natively-query-your-delta-lake-with-scala-java-and-python.htm... using Python app

"Reading the Delta Table (Python)" section towards the end of this blog refers to below code snippet

'dt = DeltaTable("../rust/tests/data/simple_table")' It is not clear if this path assumes a local folder (that would be a problem because it means I need to download delta folder to local drive)?

My delta table is on ADLS path (azure data lake store) and I do not see a way to authenticate and connect to ADLS folder path and use it in above command ( https://github.com/delta-io/delta-rs)

Am I missing something basic here?

Use Case

Querying delta table from azure function app without spinning up spark cluster

Related Issue(s)

Not sure how to auth and connect to ADLS folder where delta table is stored from the deltalake library ( I can connect using blob client but then can not use it in the DeltaTable method in deltalake package) A better code example will help

2 REPLIES 2

prasadvaze
Valued Contributor II

@denny.lee was one of the writers on this blog post so hoping he can provide a good code example

prasadvaze
Valued Contributor II

https://github.com/delta-io/delta-rs/issues/392 This issue is being actively worked on .

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group