Delta RUST API (not REST )

prasadvaze — Tue, 17 Aug 2021 17:12:27 GMT

@dennylee

Delta RUST API seems a good option to query delta table without spinning up spark cluster so I am trying out this - https://databricks.com/blog/2020/12/22/natively-query-your-delta-lake-with-scala-java-and-python.html using Python app

"Reading the Delta Table (Python)" section towards the end of this blog refers to below code snippet

'dt = DeltaTable("../rust/tests/data/simple_table")' It is not clear if this path assumes a local folder (that would be a problem because it means I need to download delta folder to local drive)?

My delta table is on ADLS path (azure data lake store) and I do not see a way to authenticate and connect to ADLS folder path and use it in above command ( https://github.com/delta-io/delta-rs)

Am I missing something basic here?

Use Case

Querying delta table from azure function app without spinning up spark cluster

Related Issue(s)

Not sure how to auth and connect to ADLS folder where delta table is stored from the deltalake library ( I can connect using blob client but then can not use it in the DeltaTable method in deltalake package) A better code example will help

Re: Delta RUST API (not REST )

prasadvaze — Tue, 17 Aug 2021 17:15:29 GMT

@denny.lee was one of the writers on this blog post so hoping he can provide a good code example

Re: Delta RUST API (not REST )

prasadvaze — Thu, 23 Sep 2021 17:07:33 GMT

https://github.com/delta-io/delta-rs/issues/392 This issue is being actively worked on .

topic Delta RUST API (not REST ) in Data Engineering

Delta RUST API (not REST )

Re: Delta RUST API (not REST )

Re: Delta RUST API (not REST )