Azure data bricks API for JSON output , displaying on UI

Mangeysh — Thu, 08 Aug 2024 14:44:59 GMT

Hello All

I am new to Azure Data Bricks and trying to show the Azure data bricks table data onto UI using react JS. Lets say there 2 tables Emplyee and Salary , I need to join these two tables with empid and generate JSON out put and calling API (end point) to display the data onto UI (react JS). I have tried to create API using Flask but it is working.Can you please suggest best way to solve this

Re: Azure data bricks API for JSON output , displaying on UI

mark_ott — Mon, 17 Nov 2025 11:54:20 GMT

The most effective way to display joined data from Azure Databricks tables (like Employee and Salary) in a React JS UI involves exposing your Databricks data through an API and then consuming that API in your frontend. Flask can work, but there are best practices and alternatives you might find easier or more scalable.

Recommended Architecture

Use Databricks (with PySpark or SQL) to join your Employee and Salary tables and generate the desired output.
Expose that output as a REST API, ideally using FastAPI (recommended for performance and modern features) or Flask.
Have your React JS frontend call this REST API, parse the returned JSON, and display it in the UI.

Steps to Achieve This

1. Join Data in Databricks

Write a Spark SQL query or use the PySpark DataFrame API in a Databricks notebook or job to join the tables:

python

from pyspark.sql import SparkSession

spark = SparkSession.builder.getOrCreate()
employee_df = spark.table("Employee")
salary_df = spark.table("Salary")
result_df = employee_df.join(salary_df, employee_df.empid == salary_df.empid, "inner")
result_json = result_df.toPandas().to_json(orient="records")

Or use Spark SQL:

sql

SELECT * FROM Employee e
JOIN Salary s ON e.empid = s.empid

You can read this into Python and output as JSON for your API.

2. Expose Result through API

FastAPI (Recommended)

FastAPI is modern, async-friendly, and integrates well with Databricks using Databricks Connect.
Example FastAPI snippet:

python

from fastapi import FastAPI
from pydantic import BaseModel
import pandas as pd
from databricks.connect import DatabricksSession

app = FastAPI()
session = DatabricksSession.builder.getOrCreate()

@app.get("/employees/salaries")
def get_employees_salaries():
    df = session.sql("""
        SELECT * FROM Employee e
        JOIN Salary s ON e.empid = s.empid
    """).toPandas()
    return df.to_dict(orient="records")

Deploy this API on Azure App Service, a VM, or Azure Functions (using HTTP trigger), whichever fits your needs and scale.

Flask (Alternative)

Flask can work similarly, but you may run into blocking or scaling issues at high concurrency. Make sure your Flask API can connect to Databricks via Databricks Connect, and confirm dependencies and network/firewall settings.

3. Call the API from React

Use axios or fetch in your React app to call the endpoint, parse JSON, and update the UI.

Example component:

javascript

useEffect(() => {
  fetch('/api/employees/salaries')
    .then(res => res.json())
    .then(data => setData(data));
}, []);

Display the data in your component as needed.

Best Practices and Tips

For production, using Databricks Connect (v2.x) with FastAPI is robust and maintainable.
Secure your API using Databricks token authentication or OAuth2.
If you need serverless or event-driven scaling, consider exposing your Databricks logic as an Azure Function with an HTTP trigger.
Prefer the Databricks SQL REST API for simple, read-only queries, but use a custom backend (FastAPI/Flask) for joined, business-logic-rich results.

Troubleshooting Flask Issues

Ensure all networking and firewalls allow inbound and outbound communication between Flask (wherever it's hosted), Databricks, and your React app.
Verify environment and dependency compatibility (PySpark, Databricks Connect, Flask versions).
Check Flask logs for stack traces and errors; often, connection issues, missing Python modules, or authentication problems are the root cause.

Resources

Microsoft Docs: Access Databricks data using external systems
Databricks Community: Building Interactive Web Apps with Databricks Connect (React + FastAPI)
Official Stack Overflow and Databricks API references for code examples and troubleshooting tips

This setup provides a scalable, maintainable, and secure pipeline from Databricks data to your Reactour React UI.

Re: Azure data bricks API for JSON output , displaying on UI

Coffee77 — Mon, 17 Nov 2025 12:00:05 GMT

How should we configure the FastAPI to access proper workspace? @mark_ott

topic Re: Azure data bricks API for JSON output , displaying on UI in Data Engineering

Azure data bricks API for JSON output , displaying on UI

Re: Azure data bricks API for JSON output , displaying on UI

Recommended Architecture

Steps to Achieve This

1. Join Data in Databricks

2. Expose Result through API

FastAPI (Recommended)

Flask (Alternative)

3. Call the API from React

Best Practices and Tips

Troubleshooting Flask Issues

Resources

Re: Azure data bricks API for JSON output , displaying on UI