cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

How to connect to a delta table that lives in a blob storage to display in a web app?

Chilangdon
New Contributor

Hi, somebody to help me how to connect a delta table with a web app? I search to a delta-rs library but I can't obtain to make the connection.

3 REPLIES 3

Anonymous
Not applicable

@Fernando Vázquez​ :

To connect to a Delta table stored in blob storage and display it in a web app, you can use the Delta Lake REST API. Here are the steps:

  1. First, make sure that you have created an Azure Blob Storage account and stored the Delta table in it.
  2. Next, create a REST endpoint in your web app that can receive requests to fetch data from the Delta table. You can use any web framework for this such as Flask, Django, or Express.js.
  3. In your endpoint function, use the Delta Lake REST API to fetch the data from the Delta table. You can make HTTP requests to the API endpoint of your Delta table with the following format:
https://<storage-account-name>.dfs.core.windows.net/<container-name>/<delta-table-name>?op=LIST&recu...

This will return a JSON response containing a list of all the Delta table files..

Once you have the list of files, you can download them and read them using any Delta Lake API. For example, you can use the delta. read_table method to read the Delta table into a Spark DataFrame.

Finally, you can serialize the DataFrame to JSON or any other format that your web app can display and return it in the response to the client.

Here's some sample Python code that shows how to fetch data from a Delta table stored in blob storage using the Delta Lake REST API:

import requests
from delta import DeltaTable
 
def fetch_data_from_delta_table():
    storage_account_name = "<your-storage-account-name>"
    container_name = "<your-container-name>"
    delta_table_name = "<your-delta-table-name>"
    api_endpoint = f"https://{storage_account_name}.dfs.core.windows.net/{container_name}/{delta_table_name}?op=LIST&recursive=true"
    
    response = requests.get(api_endpoint)
    files = response.json()["fileStatuses"]["fileStatus"]
    delta_files = [f for f in files if f["path"].endswith(".snappy.parquet")]
    
    delta_table = DeltaTable.forPath(spark, f"wasbs://{container_name}@{storage_account_name}.dfs.core.windows.net/{delta_table_name}")
    delta_df = delta_table.toDF()
    # Do some processing on the Delta DataFrame
    return delta_df.toJSON().collect()
 

This function will fetch the data from the Delta table and return it in JSON format. You can then use this data in your web app to display it to the user.

Anonymous
Not applicable

Hi @Fernando Vázquez​ 

Hope everything is going great.

Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we can help you. 

Cheers!

etsyal1e2r3
Honored Contributor

Without downloading the files directly every time, you have to create a sql warehouse cluster and connect to it via jdbc connection. This way you just use the requests library in python (or an equal one in another language like axios for javascript) and will get the response back in its format within the table. I haven't done this myself but if you do, let me know how it works out.

imagehttps://learn.microsoft.com/en-us/azure/databricks/integrations/jdbc-odbc-bi

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group