cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
cancel
Showing results for 
Search instead for 
Did you mean: 

Delta RUST API (not REST )

prasadvaze
Valued Contributor

@dennylee

Delta RUST API seems a good option to query delta table without spinning up spark cluster so I am trying out this - https://databricks.com/blog/2020/12/22/natively-query-your-delta-lake-with-scala-java-and-python.htm... using Python app

"Reading the Delta Table (Python)" section towards the end of this blog refers to below code snippet

'dt = DeltaTable("../rust/tests/data/simple_table")' It is not clear if this path assumes a local folder (that would be a problem because it means I need to download delta folder to local drive)?

My delta table is on ADLS path (azure data lake store) and I do not see a way to authenticate and connect to ADLS folder path and use it in above command ( https://github.com/delta-io/delta-rs)

Am I missing something basic here?

Use Case

Querying delta table from azure function app without spinning up spark cluster

Related Issue(s)

Not sure how to auth and connect to ADLS folder where delta table is stored from the deltalake library ( I can connect using blob client but then can not use it in the DeltaTable method in deltalake package) A better code example will help

2 REPLIES 2

prasadvaze
Valued Contributor

@denny.lee was one of the writers on this blog post so hoping he can provide a good code example

prasadvaze
Valued Contributor

https://github.com/delta-io/delta-rs/issues/392 This issue is being actively worked on .

Welcome to Databricks Community: Lets learn, network and celebrate together

Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections. 

Click here to register and join today! 

Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.