How to use databricks model serving endpoint with databricks application ?

[gokkul]

Hi DB Community ,

I have a databricks streamlit application that is viewing and inserting data into lakebase postgres database tables . Now I want to modernise this and implement databricks gen ai models . So instead of giving/selecting inputs , users can prompt what they want to insert . Prompt will be converted into sql query and data can be inserted .  So I have created a model serving endpoint . I want to know everything on how to implement this . Better if I could get any important code snippets .

[szymon_dybczak · Accepted solution]

Hi @gokkul ,

Check below databricks apps cookbook docs. The idea is to use databricks sdk inside databricks apps (or any other application) to invoke a model. So your app can ask user for an input and then pass it internally to serving_endpoints.query method.

import streamlit as st
from databricks.sdk import WorkspaceClient

w = WorkspaceClient()

response = w.serving_endpoints.query(
    name="custom-regression-model",
    dataframe_split={
        "columns": ["feature1", "feature2"],
        "data": [[1.5, 2.5]]
    }
)
st.json(response.as_dict())

Invoke a model | Databricks Apps Cookbook