Building a Delivery Assurance Agent: Predicting $8.4M in Penalty Risk with Databricks Genie

yasmeen — Fri, 28 Aug 2026 09:46:14 GMT

Delivery Assurance Agent: AI-Powered Risk Intelligence for Delivery Teams

The Problem
Delivery teams track thousands of tasks but struggle to answer:

Which client commitments will miss, and what should we do?

Traditional project tracking gives you task-level status but not commitment-level risk intelligence.

The Solution
We built the Delivery Assurance Agent - a Genie-powered platform that:
- Monitors 56 client commitments across 14 programmes
- Predicts slip probability using a 6-factor weighted risk model
- Quantifies $8.4M in expected penalty exposure
- Generates AI-drafted mitigation actions (Teams posts, Jira tickets, exec briefings)

Architecture

Data Layer: Unity Catalog (8 Tables)
- `commitment_risk` - One row per client commitment with slip probability
- `risk_factor_detail` - Six weighted factors per commitment (Throughput, Dependencies, Resources, Stakeholders, Cost, Quality)
- `milestone_forecast` - Phase gates with optimism gap detection
- `dependency_impact` - External blockers priced by commitment impact
- `leadership_actions` - Weekly decision queue
- `program_metrics` - Portfolio health (14 programmes)
- `velocity_trend` - Sprint-by-sprint throughput
- `action_queue` - Auditable log of generated artifacts

Intelligence Layer: Genie Space
- 13 benchmark queries
- 8 curated starter questions for executives
- Entity matching on 28 columns
- 60+ documented columns with business context
- SQL examples teaching risk model patterns

Application Layer: Databricks Apps V2
- FastAPI backend with service principal execution
- Genie API integration for conversational queries
- SQL Warehouse for KPI dashboard
- Model Serving (Llama 4) for action generation
- OAuth2 for user context

The Risk Model

Six Weighted Factors (Logistic Regression):
1. Throughput(1.75) - Current velocity vs. remaining work
2. Dependencies(1.20) - External blockers and late deliverables
3. Stakeholder Signal (0.80) - Client escalations, team concerns
4. Resources(0.75) - Staffing gaps, attrition
5. Cost Performance(0.70) - Budget overruns (CPI < 1.0)
6. Quality (0.62) - Defect rates, rework cycles

Intercept: -3.2

Formula: `slip_probability = logistic(Σ(factor × weight) - 3.2)`

Accuracy: 92% on 25 delivered commitments (50% threshold, 5-day tolerance)

Key Features

1. Executive Intelligence

Ask: "What is the total expected penalty exposure?"
- 31 commitments at risk
- $8.4M expected exposure (probability-weighted)
- $18.9M worst-case (if all miss)
- Average 52% slip probability

2. Root Cause Analysis
Ask: "Why is the Atlas Core Banking Migration at risk?"
- Throughput: 47% contribution (velocity can't close remaining work)
- Dependencies: 31% contribution (three teams late)
- Named drivers, quantified shares, responsible owners

3. Optimism Gap Detection

Ask: "Where are reported dates not supported by data?"
- Compares forecast_date (what PM reports) vs. predicted_date (what data implies)
- Surfaces commitments where optimism gap > 7 days

4. AI Action Generator
The app generates data-grounded artifacts:
- Teams notifications: Bold headline, 3-4 evidence bullets, clear ask
- Jira tickets: JSON payload with priority, assignee, acceptance criteria
- Executive summaries: Board-ready briefing (under 200 words)
- Mitigation plans: Sequenced actions with owners and dates

All artifacts logged to `action_queue` with the model endpoint that produced them.

Databricks Features Used
1. AI/BI Genie - Natural language to SQL
2. Unity Catalog - Governed 8-table data model
3. SQL Warehouses - Serverless query execution
4. Databricks Apps V2 - Production app hosting
5. Databricks SDK - Programmatic Genie + SQL integration
6. Model Serving- AI action generation (Llama 4)
7. Service Principals - Consistent app-level permissions

Results
- ✅ 100% benchmark pass (13/13 queries)
- ✅ 60+ columns documented with entity matching
- ✅ 8 starter questions for immediate value
- ✅ Responsible AI: "AI-generated risk assessment. Verify before action."
- ✅ Production-ready: Service principal execution, auditable queue

Key Takeaways
1. Genie as a Platform: Not just Q&A - integrate with KPIs, actions, and workflows
2. Model Interpretability: Six named factors beat a black box
3. Probability-Weighted Exposure: Rank by expected_penalty × slip_probability, not worst-case
4. Responsible AI: Always include verification guidance for high-stakes decisions

Try It Yourself
The pattern is reusable for any domain with:
- Commitments/deadlines you must hit
- Multiple risk factors you can measure
- Financial or reputational consequences of missing

Links
- Live App: [View Demo](https://delivery-assurance-agent-7474648612775687.aws.databricksapps.com)

Re: Building a Delivery Assurance Agent: Predicting $8.4M in Penalty Risk with Databricks Genie

rajkumar8k — Thu, 03 Sep 2026 09:09:55 GMT

Excellent article

topic Re: Building a Delivery Assurance Agent: Predicting $8.4M in Penalty Risk with Databricks Genie in Data Engineering

Building a Delivery Assurance Agent: Predicting $8.4M in Penalty Risk with Databricks Genie

Re: Building a Delivery Assurance Agent: Predicting $8.4M in Penalty Risk with Databricks Genie