Commentary functionality through Databricks

[Shivaprasad]

In on-Prem we currently create dashboards which basically provides year over year or quarter over quarter comparison. When the variance is more than certain threshold for a particular data intersection business has the option to add comments, also someone else can review/approve the comments. We provide a link in the dashboard which takes the business user to the commentary page(developed in Java) when the variance is more than the threshold limit.

Do we able to provide similar functionality through Databricks.

[nayan_wylde]

There is no inbuilt features to built-in “commentary workflow” like your Java app. But you can custom build one. Here are the steps to build one.

1.  Use Databricks SQL Dashboards or Lakeview Dashboards for your YoY/QoQ variance analysis.

2. Create a Delta table to store comments, approvals, and metadata. Something like this.This table can be secured with UC for RBAC (who can comment, who can approve).

%sql

CREATE TABLE commentary (
  id BIGINT GENERATED ALWAYS AS IDENTITY,
  metric STRING,
  period STRING,
  variance DOUBLE,
  comment STRING,
  status STRING, -- e.g., 'Pending', 'Approved'
  created_by STRING,
  created_at TIMESTAMP,
  approved_by STRING,
  approved_at TIMESTAMP
);

3. Use Databricks SQL  to expose the data intersection (metric + period) where variance > threshold.

4. Build a custom UI (React/Next.js or Streamlit on Databricks) that:
         --- Lists flagged variances.
         ---Allows adding comments.

5. In Lakeview Dashboards, you can add a custom URL action:

       ----Example: https://your-commentary-app?metric={{metric}}&period={{period}}

6. Use Databricks Workflows or Jobs to:

     ---Trigger alerts when new variances exceed thresholds.

     ---Send email/Slack notifications for pending approvals.