Genie Hub topics

How to make each person see only their specific data (i.e. their own rows in Genie) with RBAC & ABAC

Valeria_Koz_DBX — Tue, 29 Sep 2026 17:01:50 GMT

When rolling Genie out to a UK enterprise Finance department, one of the questions was "will a cost-centre owner accidentally see another team's numbers?"

In Finance, one leaked row is a bigger problem than a slightly clumsy chart. The good news: you don't secure Genie separately. Genie runs SQL through Unity Catalog, so it inherits whatever access rules you set on the data. The trick is knowing which of the two levers to reach for — and one configuration step that decides whether either of them works at all.

The mental model: RBAC gets you to the table, ABAC gets you to the right rows

RBAC (role-based) answers who can touch this object at all. You grant a role/group access to a catalog, schema, or table. Coarse-grained, and where you should always start.
ABAC (attribute-based) answers which rows inside that table this person may see. It's driven by attributes — the user's group or region, and tags on the data — evaluated at query time. Fine-grained, and what actually delivers "each person sees only their rows."

Genie respects both automatically — but only if it runs queries as the viewer, not as the room's creator. That one setting is the whole ballgame.

Step-by-step

Step 1: Start with RBAC — grant table access by role. Give the business group access to the objects the Genie room needs, and nothing more.
GRANT SELECT ON TABLE finance.actuals.cost_centre_spend TO `finance_analysts`;
Step 2: Make the Genie space run on-behalf-of the viewer (OBO). This is the make-or-break step. Enable user authorization / identity forwarding so queries execute under the signed-in user's identity. If the space runs as its creator or a single service principal, Unity Catalog only ever sees that one identity — so everyone sees the creator's rows. Configure the space (or backing app) to forward the user's SQL scope rather than running as a fixed principal.
Step 3: Model the attributes. Decide what row visibility keys off — usually group membership (RBAC-style) or an entitlement table mapping each user to their cost centres/regions.
-- who is allowed to see which cost centre
CREATE TABLE finance.security.cost_centre_acl (user_email STRING, cost_centre STRING);
Step 4: Apply the row filter. Two routes:

4.1 Classic (UDF-based): a small function that checks the session identity, attached to the table.
CREATE FUNCTION finance.security.cc_filter(cc STRING)
RETURNS BOOLEAN
RETURN is_account_group_member('finance_admins') -- admins see all
OR EXISTS (SELECT 1 FROM finance.security.cost_centre_acl
WHERE user_email = current_user() AND cost_centre = cc);

ALTER TABLE finance.actuals.cost_centre_spend
SET ROW FILTER finance.security.cc_filter ON (cost_centre)

4.2 ABAC (tag-driven, GA 2026):

tag the sensitive column with a governed tag once, then let a single row-filter/column-mask policy apply everywhere that tag appears — far less per-table plumbing at scale. (Confirm the exact policy DDL and any workspace-specific limits in your own environment before rollout.)

5. Step 5: Prefer OAuth over PATs, and viewer credentials over embedded. Personal access tokens and embedded creds quietly bypass per-viewer identity — they're the most common way RLS "silently stops working."

6. Step 6: Test as two personas. Open the room as two different users and ask Genie the same question ("show my cost-centre spend this quarter"). The rows must differ. Cross-check against the audit log that the query ran under the viewer's identity, not the creator's.

(Screenshot placeholder: side-by-side of the same Genie question asked by two users, returning different rows.)

What good looks like (expected outcome)

Two people ask Genie the identical question and each gets only their own rows — with no filtering logic living inside Genie itself. Security stays in Unity Catalog, so it holds no matter how anyone phrases the question, and new tables inherit protection the moment the governed tag is applied.

Common errors (and fixes)

Room runs as creator/service principal → everyone sees the creator's rows. Fix: enable OBO / viewer identity.
PAT or embedded credentials in the path → identity doesn't flow, filters don't apply. Fix: OAuth + viewer credentials.
Filtering in a dashboard/app layer instead of UC → Genie bypasses it entirely. Fix: push the rule down to a row filter/ABAC policy
Overlapping ABAC policies on the same table/principal → access gets blocked outright. Fix: keep policies non-conflicting and test the matrix.
Ungoverned tags → ABAC policies won't reliably bind. Fix: use governed tags.

Takeaway: you never secure Genie — you secure the data, and let Genie inherit it. RBAC gets people to the table; ABAC gets them to the right rows; running as the viewer is what makes both real.

How to get people to actually choose Genie: train the mindset, not just the buttons

Valeria_Koz_DBX — Mon, 28 Sep 2026 17:34:55 GMT

The hardest part of our Genie rollout wasn't building the room. It was 9am the next morning — when people still opened Excel.

We rolled out a Genie Space to a UK enterprise Finance function (heavy excel users). The room was scoped well and the answers were good. But adoption wasn't rising — and not because the platform was hard. It was stalling because we needed to train people properly, not only how to use Genie, but also how to think in Genie.

The gap was a mental model.

Excel filtering is deterministic: you already know the rows you want, so you slice down to them.
Genie is a conversation. You ask a colleague a question that comes back, and refine— a follow-up, a correction, a "now break that down by region." People who keep the Excel model try and give up when the first answer isn't exact. People who treat Genie as a colleague they iterate with get there in three turns.

So our training had to cover three things, not one: how to use the platform, how to prompt, and how to change the mental model. Here's the sequence that worked.

Step 1: Lead with the mindset, not the UI. Open every session with the reframe — Genie is not a search box or a filter; it's a colleague you ask, then push back on. Spend the first ten minutes here, before anyone touches the screen. If people keep the filtering model, no amount of UI training sticks.
Step 2: Show the magic before the mechanics. Start with one genuinely impressive answer to a question they actually ask on a Monday morning — before you explain how it works. The "how" only lands once they want it.
Step 3: Teach prompting as a skill. Three habits: be specific (name the measure, the period, the grouping), give context, and — above all — iterate. Model a real exchange live: a vague first question, a mediocre answer, then two follow-ups that sharpen it. The lesson people need to see is that the follow-up is the skill, not the opening question.

Show them exactly what "iterating like you would with a colleague" looks like (check the screenshots below)

You: What is our monthly actual cost?
Genie: Monthly spend for the past year
You: Can you split each month with subcategories and present in the stacked bar view?
Genie: [returns a bar chart view]
You: Can you do a month-by-month comparison on different categories
Genie: [returns the month-by-month change per category]

4. Step 4: Split the group into breakout rooms (Small groups of 7–9). Big enough for discussion, small enough that
everyone can present their question in the room. People don't change a habit by watching a demo; they change it by doing it once, with help

5. Step 5: Let peers teach the next cohort. The person who "got it" this week becomes part of the champion's network and teaches/presents in next session. A colleague saying "I used to export to Excel too" moves more people than any expert — and it scales the rollout without a central team as the bottleneck.
the bottleneck.

6. Step 6: Anchor the new habit to the old trigger. Find the specific task the old tool owned at 9am, and rehearse doing that exact task in Genie. A habit only changes when the new thing beats the old thing at the moment you'd have reached for it.

What good looks like (expected outcome)

By the end of one session, every participant has done the full loop themselves at least once: asked a real question, hit a mediocre answer, and recovered it with a follow-up — without you touching their keyboard. That single recovery is the moment the mental model flips. Within a couple of weeks you should see people opening Genie for the recurring Monday-morning question they used to export to Excel for — that's the real success metric, not how many people attended the training.

Common errors (and how to fix them)

Chasing one perfect prompt. People write a long, exact question, get a wrong answer, and quit. Fix: ask something short, then refine — the follow-up is the tool, not the opening line.
Prompts too vague. "Show me sales" gets a vague answer. Fix: name the measure, the period, and the grouping.
Reverting to Excel the moment an answer looks off. Fix: correct Genie in-thread, the way you'd correct a colleague ("no, I meant net not gross") — don't abandon the conversation.
Teaching the buttons first. UI-led training leaves the Excel mindset intact. Fix: mindset first, mechanics second.
Training a big room with no opportunity for each person to try it out with others. In a room of 30, most people never type anything. Fix: keep it to 7–9 so everyone actually asks a question.

The takeaway: a Genie Room only creates value when someone chooses it over their old habit. That's won in how you teach, not in what you build — and the thing you're really teaching isn't the product. It's a new way of asking.

DataSmart

fawad956 — Mon, 28 Sep 2026 17:17:21 GMT

For about a year, I was the reason half my team's questions took three days to get answered, and I didn't realize I was the bottleneck until someone said it to my face.

Every ad hoc data request went through me. Someone needed a number for a Friday meeting, they submitted a request, it sat in a queue behind four other requests, and by the time I got to it the meeting had usually already happened. I thought that queue was proof the data was being handled carefully. What it actually did was teach people to stop asking me and start building their own spreadsheets from whatever export they could get their hands on, which is a worse outcome for accuracy than the delay I was trying to prevent.

What I got wrong: I equated being the single path to an answer with being careful. They're not the same thing. Being the only person who can answer a question doesn't make the answer more trustworthy, it just makes the business slower while looking responsible.

What I know now, and what a Databricks report on enterprise AI described almost exactly: a Fortune 50 manufacturer with more than nine million data assets hit this same wall at a much bigger scale, tens of thousands of employees needing access, one small central team standing between all of them and an answer. Their fix wasn't more people in the queue, it was governed self-serve access, so the central team stopped being the path to every answer and started being the reason self-serve was actually trustworthy.

→ A backlog isn't proof of rigor, it's usually proof the access model is wrong.
→ People don't stop needing answers when you're the bottleneck, they just start getting worse ones somewhere else.
→ The real job shifted from answering every question myself to making sure the answers other people could get to on their own were actually right.

Have you ever realized you were the bottleneck in a process you thought you were protecting?

Trust in Genie isn't won with accuracy — it's won with a visible feedback loop accountability

Valeria_Koz_DBX — Tue, 22 Sep 2026 16:34:26 GMT

A Finance team won't trust a Genie Room just because it's right. They trust it when they see someone is accountable for the answers.

What worked for us:

a thumbs up/down on every chat, and total transparency about who reads it. We had a dedicated Finance team owning the rollout — they knew the real questions and investigated every miss.
and a thumbs-down doesn't just sit in a dashboard; it goes to the product team, and Databricks genuinely comes back to you on it.

That's the shift: trust isn't won with accuracy stats — it's won when people see a loop. My feedback → a real person → a real change.

How do you close the feedback loop on your Genie rooms — and who owns the answers? 👇

#Databricks #Genie #ChangeManagement

State of AI Agents

fawad956 — Mon, 21 Sep 2026 14:20:05 GMT

Three years ago, provisioning a test database at work meant a full ticket cycle with my name on the approval. Last week I watched an analyst spin one up in under a minute using an AI agent, no ticket, no wait.

I did not expect database administration to be the first place I saw agents take over.

A new Databricks report says AI agents now create 80 percent of new databases and 97 percent of test branches industry wide, up from near zero two years ago.

I used to be the checkpoint for that process. Now I am not the checkpoint, I am the one deciding what the agent should never be allowed to touch without a human looking first.

The job did not disappear. It moved somewhere higher up the stack.

If you work near data infrastructure, has your role moved up the stack too, or is it still mostly the old work?

Building Deterministic Databricks Genie Agents

mehulbhuva — Thu, 10 Sep 2026 16:14:55 GMT

AI can convert business questions into SQL, but it cannot reliably infer your business logic. Learn how to design a Databricks Genie Agent that delivers accurate, deterministic answers instead of making assumptions.

Drawing from 22+ years of enterprise IT/Data experience, I’ve put together a 4-layer blueprint to build reliable, production-grade Databricks Genie Agents (formerly called Genie Spaces):

• 📊 Data: Pre-joined Unity Catalog views to eliminate guessing.
• 📝 Instructions: Explicitly documented tribal knowledge.
• 🔢 SQL Expressions: Locked-down, certified metrics.
• ⚙️ Example Queries: Seeded Q&A pairs for complex logic.

Stop answering the same ad-hoc questions. Shift your role from query writer to knowledge architect.

👉 Read the full guide here: https://www.sqlservercentral.com/articles/databricks-genie-spaces-for-sql-analysts-natural-language-querying-without-leaving-your-data-platform

Why we didn't build one big Genie Room

Valeria_Koz_DBX — Wed, 09 Sep 2026 17:02:32 GMT

When we started, the obvious move looked like one big Genie Room — put everything in, let people ask anything. We went a different way, and it's held up, so I wanted to share the reasoning rather than the conclusion.

This was for a UK enterprise Finance function, and we split it in two:

Narrow, trusted rooms. Two rooms — one operational, one financial — each pointed at a single objective. A Genie Room is at its best around 7 tables; it copes with 30; it struggles at 100. Keeping each room tight is what makes the answers trustworthy.
One blended, exploratory room. A separate space across finance, operations and HR (SAP, SAP Analytics Cloud, Workday) for the cross-functional questions no single system can answer alone.

The rule we landed on: let the question set the scope — narrow when you need trust, broad when you're exploring. And put access controls in at the start, not as a retrofit.

Curious how others are drawing that line.

Could Databricks app support the integration of Genei Room and Power BI?

fjrodriguez — Tue, 08 Sep 2026 10:46:16 GMT

Is it possible: a Databricks App embedding Power BI ? Rather than placing Genie inside Power BI, place Power BI inside Databricks. A Databricks App (React or Streamlit) hosts native Genie chat alongside a Power BI report embedded via the JavaScript SDK.

Databricks App using the AppKit Genie plugin (GenieChat, useGenieChat, SSE streaming, conversation persistence) or the SDK directly
Power BI Embedded in embed-for-your-organization mode (user-owns-data) with Entra ID
App registration in Entra and workspace permissions in Power BI
Hosting and application lifecycle ownership

The hardest part of Genie rollout isn't building it, it's changing what people reach for at 9am. 🧠

Valeria_Koz_DBX — Mon, 07 Sep 2026 16:39:35 GMT

You can build a perfect Genie Space — clean metrics, tuned instructions, the right tables — and still watch people open their old spreadsheet at 9am out of habit. The room isn't the hard part. The habit is.

I've been running a Genie rollout inside a UK enterprise Finance function, and the thing that actually moved daily usage had almost nothing to do with what we built. It was how we taught it. Four things made the difference:

Teach prompting, not the UI. The button tour lands flat. What sticks is conversational skill — how to ask, how to follow up, how to narrow a vague question into a good one. That's the transferable capability; the interface takes care of itself.
Show the magic before you explain the mechanics. Lead with a genuinely impressive answer to a question people actually care about. Earn the "wow" first — curiosity opens the door that a features walkthrough never will.
Keep the groups small. Breakouts of 7–9, hands on keyboards. People won't ask the "silly" question in a room of 30, and it's the silly questions where confidence is built.
Let peers teach peers. The moment a colleague shows another colleague something they found themselves, adoption stops being a rollout and starts being a habit. Your job is to create those moments, not to be the only teacher in the room.

The underlying shift: a room only creates value when someone chooses it over their old habit — and that choice is won in how you teach, not in what you build. We treated enablement as a behaviour-change problem, not a training tick-box, and that's what changed the 9am reflex.

Would be interesting to know how others have handled this: when you roll out Genie, where does most of your effort go — into the room, or into changing the habit around it? And what actually shifted daily usage for you?

Announcement | Operationalizing Genie Ontology in Your Data Stack

Tushar_Parekar — Fri, 04 Sep 2026 10:38:56 GMT

Genie Ontology helps Genie One and Genie Agents use business context from governed data, semantic definitions, and existing work assets. The most practical way to improve answer quality is to strengthen that foundation progressively, one business domain at a time.

A six-layer path to better answers

Layer 0 | Build a solid data foundation: Start with clean tables, clear schemas, reliable business grain, and consistent entities. Resolve duplicate identities and shape trusted consumption surfaces before asking agents to reason across the estate.
Layer 1 | Enrich metadata: Add useful table descriptions, column comments, and governed tags in Unity Catalog. Tools such as dbxmetagen can help generate a first pass, while human review keeps the final metadata accurate.
Layer 2 | Model business meaning: Define relationships, critical metrics, dimensions, synonyms, and formatting with Unity Catalog metric views. Use Domains to organize assets and Pages to document authoritative business concepts.
Layer 3 | Curate context-rich assets: Make dashboards, notebooks, queries, and Genie Agents useful sources of context by documenting them, adding examples, and keeping them current. Certification and deprecation help signal which assets are trusted and which should no longer guide users or agents.
Layer 4 | Apply governance: Use Unity Catalog permissions and fine-grained controls to govern access to data and context. Unity AI Gateway adds a central control point for AI services, model access, policies, and usage.
Layer 5 | Evaluate and improve: Create realistic questions for each priority domain, compare answers with known results, inspect generated SQL and citations, and use feedback to fix the right layer. Genie Agent monitoring and benchmarks provide a repeatable way to track quality and usage over time.

👉 Read the full post here

Announcement | Get Started with Genie One: Top AI Cowork Use Cases for Business Users

Tushar_Parekar — Wed, 02 Sep 2026 15:25:57 GMT

Genie One helps business teams turn everyday, repetitive work into repeatable workflows using governed data, connected tools, and natural language.

Key highlights

Automate business reviews: Use a standard format, pull in current data, add commentary, and schedule recurring reports.
Simplify meeting prep and follow-up: Create meeting briefs, summarize decisions and action items, and organize follow-up work.
Keep documents up to date: Draft and refresh policies, playbooks, FAQs, and other documents using approved templates and business data.
Monitor important signals: Track metrics such as revenue, ticket volume, job failures, or inventory and receive alerts when something changes.
Save workflows as skills: Turn useful instructions into reusable skills that teams can run again or schedule for regular tasks.

The best place to start is with one recurring task that takes time every week. Connect the right data sources, give Genie One clear instructions, review the first few results, and refine the workflow as needed.

👉 Read the full post here 👈

Announcement | Beyond answers: New Genie One features to turn insights into action

Tushar_Parekar — Thu, 03 Sep 2026 12:45:23 GMT

Databricks is expanding Genie One beyond answering questions, helping business users move from insight to action with a desktop app, collaborative documents, reusable agents, and richer enterprise context.

What’s new

Genie One on the desktop: The new macOS desktop app keeps Genie One accessible through a global launcher, with conversations that continue across the web and desktop experiences.
Create and collaborate on documents: Turn a Genie One conversation into an editable document, add comments, review changes, view version history, and share a read-only link with colleagues.
Reuse work with Genie Agents: Save the context from a conversation as a Genie Agent, refine its instructions and data assets, and share it with teammates for repeatable workflows.
Share insights without starting over: Send colleagues a read-only link to a chat, including its questions, answers, visualizations, and context.
Bring more context into conversations: Genie One can use governed business context from Genie Ontology, analyze uploaded CSV, Excel, PDF, image, and Word files, and connect to external sources such as Google Drive, SharePoint, GitHub, Glean, Slack, and Atlassian.
Take action with connected tools: Some MCP connections can perform write actions in external systems, with available actions determined by the tool, consented OAuth scopes, and the user’s permissions.

👉 Read the full post here 👈

After Databricks Summit 2026, I Feel Data Engineering Is Entering a New Phase

Brahmareddy — Wed, 02 Sep 2026 14:42:10 GMT

Dear Databricks Community, After coming back from Summit 2026, one thought stayed in my mind.

Data engineering is changing again. Earlier, most of our work was around pipelines, tables, jobs, transformations, reports, and dashboards. All of that is still important. But now, data is moving much closer to decisions, applications, AI, and real-time actions.

A few topics stood out to me strongly: LTAP, Lakebase, Genie, and Lakehouse RT.

LTAP made me think about how analytics and transaction processing are coming closer. For a long time, applications handled transactions and analytics systems handled reporting. Then we moved data between them using batch, streaming, CDC, or ETL pipelines.

But today, users want fresh insights. Applications need faster intelligence. AI systems need current context. So the gap between operational data and analytical data needs to become smaller.

Lakebase also feels like an important step. To me, it is not just another database. It shows how database architecture is evolving toward the lakehouse. If operational data can work more closely with trusted lakehouse data, it can reduce duplication, simplify architecture, and open new patterns for Data + AI applications.

Genie feels very practical too. Dashboards are useful, but they usually answer only the questions we already planned for. Business users always have follow-up questions. Why did this number change? Which team, region, or process is driving it? What happened this week?

Natural language analytics can make this easier. But one thing is clear: AI can only answer well when the data foundation is strong. Clean data, trusted metrics, governance, permissions, and context still matter a lot.

Lakehouse RT is another exciting direction. Earlier, real-time data was needed only for special use cases. Now, many business problems need faster updates. Fraud, inventory, customer experience, monitoring, recommendations, operations, and AI agents all need fresh data.

The real question is not only, “Can we process the data?”

It is, “Can we process it while it is still useful?”

My biggest takeaway from Summit 2026 is simple.

The future data platform is becoming more connected. Transactions, analytics, AI, applications, and real-time data are coming closer.

This means data engineers need to think beyond pipelines and reports. We need to think about the full solution.

What problem are we solving?
Who will use this data?
How fast do they need the answer?
Can AI make this workflow easier?
Can real-time data improve the decision?

For me, this is the exciting part of working with Databricks. A small idea can become a POC. A POC can become a dashboard. A dashboard can become a real-time insight. A real-time insight can become an action.

That is how I see the next phase of data engineering. It is not only about moving data faster. It is about helping people make better decisions sooner.

After Summit 2026, which area are you most excited to explore with Databricks: LTAP, Lakebase, Genie, Lakehouse RT, or another Data + AI use case?

Operationalizing a Live Genie Space: Benchmarking, Governance & Continuous Improvement

Salman_Ahmed — Tue, 01 Sep 2026 20:25:33 GMT

In my previous article, I wrote an article about building a strong foundation for Databricks Genie Spaces through data modeling, metadata, and semantic design. This time, I want to share a few lessons from running a Genie Space against a real healthcare analytics use case.

The biggest surprise wasn't AI. It was how quickly Genie exposed problems that already existed in the data.

When we first deployed the Genie Space and ran it against our benchmark questions, the results were far from perfect. The out-of-the-box configuration answered only 6 out of 20 benchmark questions correctly. Even after multiple rounds of tuning, a repurposed legacy analytics dataset for BI workloads never achieved more than 75% benchmark accuracy.

What changed everything wasn't prompt engineering. It was dataset design.

Once we defined and built a live delta dataset with pre-defined metrics specifically for Genie, removed irrelevant columns, simplified business logic, standardized categorical values, and aligned the schema to the questions users were actually asking, benchmark accuracy eventually reached 100%.

While many teams may focus on configuring Genie and fine-tuning prompt. For our team, the biggest boost in accuracy came from preparing the underlying data for Genie.
Another lesson was the importance of benchmarking. Without benchmarks, every discussion becomes subjective.
- Quick Tip: Users may ask the same question in different ways. Databricks recommends using 2-4 different phrasings of the same question (but the same SQL code) to fully assess accuracy.

We also learned that not every configuration change improves accuracy. Below is a visual of our findings;

Table descriptions helped significantly.
General instructions provided some improvement.
Surprisingly, adding large numbers of synonyms wasn't always beneficial and sometimes introduced additional ambiguity.
The most effective configuration element by far was SQL examples.
- Quick Tip: Whenever a business rule involved non-trivial logic, such as event-level calculations, eligibility definitions, or classifications, SQL examples consistently outperformed additional instructions. Genie seemed to learn far more effectively from concrete examples than from lengthy explanations.
Another important observation was that Genie is not entirely deterministic. The same question can occasionally be presented differently depending on formatting choices, percentage calculations, null handling, or interpretation of date ranges. We found ourselves spending as much time standardizing outputs as improving answers.

This is where governance becomes essential

A successful Genie Space needs more than good metadata. It needs agreed definitions, controlled business logic, benchmark testing, and a clear process for evaluating changes. Otherwise, the results erodes trust.

The teams that will get the most value from Genie Spaces and turn a demo into a production-ready analytical product won't be the ones writing the most sophisticated prompts. They'll be the ones investing in benchmark-driven development, semantic consistency, and Genie Tailored purpose-built datasets.

Making Databricks Genie Spaces Actually Work: A Practical Framework for Client and Data Teams

Salman_Ahmed — Tue, 01 Sep 2026 18:42:28 GMT

Introduction

Over the past year, I've had quite a few conversations with teams exploring Databricks Genie Spaces. The pattern is usually the same. Someone sees a demo, watches a business user ask a question in plain English, and within seconds Genie returns a chart, a SQL query, and what appears to be a perfectly reasonable answer.

The reaction is almost always immediate.

"This could completely change how people use data and BI reports."

For years we've built dashboards, semantic models, reporting layers, and self-service analytics platforms with the goal of helping business users answer questions faster. Genie feels like the natural next step in that journey. Instead of learning SQL or navigating dozens of dashboards, users can simply ask a question and interact with data conversationally.

The technology itself is impressive. But after working on enterprise data platforms for many years, the challenge is rarely the AI model itself.

The challenge is trust.

Can users trust the answer? Can analysts reproduce it? Can data teams explain it? And perhaps most importantly, will different users receive consistent answers to the same question?

Those questions have far less to do with the language model and much more to do with the foundation underneath it. That's why whenever I'm asked how to improve Genie Spaces.

I rarely start by talking about prompts, I start by talking about data.

Why Most Genie Projects Fail Before Users Ask Their First Question

When teams evaluate Genie Spaces, their first instinct is often to improve prompts or add more instructions.

In my experience, that's usually the wrong starting point. Most quality issues originate from one of five areas:

Weak data modeling (Data Engineering)
Poor metadata quality (Data Scientist/Business Analyst/Data Analyst)
Undefined business metrics (Data Scientist)
Missing table relationships (Data Engineering)
Lack of benchmark testing (ML Engineer)

Databricks has been investing heavily in a governed semantic foundation through Unity Catalog Semantics, Genie Ontology, Live Tables, including metric views, domains, governed business definitions, and AI-aware context management. These capabilities help ensure that both humans and AI systems interpret data consistently.

Step-by-step plan

Step 1: Build the Data Foundation Before Building the Genie Space

The single most important success factor is the quality of the curated data layer.

Many data teams expose highly normalized source models and expect Genie to figure out the relationships. While technically possible, this often introduces ambiguity. Instead, design datasets specifically for consumption.

Recommended Design Approach

1. Denormalize Where Appropriate

Rather than expecting Genie to navigate a maze of joins every time a user asks a question, it's worth investing in curated business-ready delta tables. If answering a simple revenue question requires six or seven tables to be stitched together, the chances of selecting an incorrect relationship increase significantly. In most successful implementations I've seen, common dimensions are already joined, business entities are standardised, and duplicate relationship paths have been removed long before the data reaches Genie.

2. Pre-Calculate Common Business Logic

A common mistake is treating Genie as the place where business logic should be assembled. In reality, repetitive calculations and classifications belong in the data layer. Whether it's reporting periods, fiscal calendars, active customer definitions, or product lifecycle states, these concepts should already exist in a governed and reusable form. This allows Genie to focus on answering the question rather than reconstructing business logic every time.

3. Establish Canonical Metrics

One of the strongest capabilities available through Unity Catalog is the ability to define reusable metrics and semantic objects that provide consistent business logic across analytics workloads and AI consumers.

For example:

measures: - name: Total Revenue expr: SUM(purchase_amount) FILTER (WHERE status='approved') comment: Revenue from approved transactions display_name: Total Revenue synonyms: - revenue - sales - total sales - approved revenue

This ensures that every user asking about revenue receives answers based on the same calculation.

Step 2: Treat Genie Like Software and Create Benchmarks

I've been in sessions where a team asks Genie three questions, gets two correct answers, one questionable result, and immediately starts debating whether the prompt needs to be rewritten. The reality is that this kind of testing is far too subjective. Without a defined set of benchmark questions and expected outcomes, it's almost impossible to measure quality in a meaningful way. That's why it is best to establish a benchmark suite early, before wider adoption begins.

Create a Question Inventory

The best benchmark questions usually come directly from the people who use the data every day. Spend time with business stakeholders, analysts, and subject matter experts to understand the questions they regularly ask, whether that's tracking KPIs, understanding trends, explaining variances, or preparing executive reporting. Once you've collected those questions, document what a correct answer looks like. That includes not only the expected result, but also the level of aggregation, any business filters that should be applied, and how the answer should be presented. The goal isn't simply to test whether Genie returns an answer. It's to verify that the answer aligns with how the business expects the question to be interpreted.

Build a Regression Test Suite

A Genie Space is never really finished. The underlying data platform keeps evolving, new business requirements appear, and teams continuously refine their definitions and metrics. While those changes are important, they also introduce risk. I've found that the most successful teams maintain a set of benchmark questions that are executed regularly, especially after major updates. It provides a simple but effective way of confirming that answers users already trust continue to behave as expected, even as the platform grows and changes around them.

Step 3: Teach Genie How Your Business Thinks

Metadata is what helps bridge the gap in business thinking in natural flow and tables, columns, or schemas. The richer the business context around your data, the easier it becomes for Genie to understand what the user is really asking and translate that intent into a query that makes sense. In many cases, improving metadata delivers a bigger uplift in answer quality than yet another round of prompt tuning.

Table Descriptions Matter

Avoid generic descriptions like:

Customer transaction table

Instead use:

Contains finalized customer purchase records used for revenue reporting and financial performance analysis.

The second description provides significantly more business context.

Define Synonyms Explicitly

Business users rarely use technical column names.

For example:

Business Term Actual Field

Sales	Revenue
ARR	Annual Recurring Revenue
Customer Base	Active Customers
Gross Sales	Invoice Amount

Providing synonyms dramatically improves question interpretation.

Document Relationships

Another area that often gets overlooked is the way datasets relate to one another. In most enterprises, the same business entity appears across multiple tables, and there can be several possible paths between them. If those relationships aren't clearly defined, Genie may have to infer how the data is connected, which can lead to unexpected results.

Explicitly documenting relationships and validating the business meaning behind them significantly improves consistency. It's not enough to know that two tables can be joined; Genie also needs to understand how they should be joined and what business context that relationship represents.

Incorrect joins are a major source of AI-generated analytical errors.

Supply Example SQL

One of the most effective yet underutilized techniques is maintaining a library of gold-standard SQL.

Example:

SELECT fiscal_year, SUM(revenue) AS total_revenue FROM sales_gold GROUP BY fiscal_year ORDER BY fiscal_year;

These examples act as patterns that help Genie generate more reliable queries.

Use General Instructions Sparingly

Many teams attempt to solve every issue through lengthy instructions.

This typically creates maintenance problems.

A simpler decision framework is:

Problem Fix Location

Wrong table	Table metadata
Wrong column	Column description
Wrong value mapping	Example values
Wrong join	Relationship definition
Wrong calculation	Example SQL

Use narrative instructions only for business context.

Key Takeaways

Organizations often assume conversational analytics starts with AI.

In reality, it starts with data engineering.

Before focusing on prompts, invest in:

Curated Gold datasets
Metric definitions
Rich metadata
Relationship modeling
Benchmark testing

Genie Spaces are most successful when they are grounded in governed business semantics rather than isolated prompt instructions. Databricks' broader investment in Unity Catalog Semantics reflects this exact direction, creating trusted business context that can be reused across analytics and AI experiences.

2-Part Series

Part 1: Making Databricks Genie Spaces Actually Work: A Practical Framework for Client and Data Teams
Part 2: Operationalizing a Live Genie Spaces with Benchmarking, Governance, and Continuous Improvement

Money Performance and customer Insights

amitsharma1707 — Tue, 25 Aug 2026 20:23:47 GMT

Problem / opportunity
Lending teams wait days for analysts to answer routine portfolio questions. The data is already modeled — the bottleneck is access. This app puts a conversational agent on a governed lending gold layer so answers come back in seconds using certified metric definitions.

Who it's for
Credit risk and portfolio managers, collections leads, underwriting and credit policy teams, regional and product heads, and lending executives — business users who need answers but don't write SQL.

Architecture and data flow
Medallion on the lakehouse. Bronze lands core lending, servicing/payments, bureau, and origination data as-is. Silver cleanses and conforms it into dimension tables. Gold serves four curated products: Br 360, Performance Mart, Delinquency & Aging, and Origination & Channel. A Genie space scoped to gold only powers the app; Unity Catalog governs access and row-level security throughout.

What users can ask
Portfolio health (PAR30, default and collection rates, NPL counts), profiles (credit grades, segments, exposure), origination trends (by product, officer, region, vintage), and payment behavior (installment collections, delinquency patterns). Anything outside the lending book is out of scope.

How Genie powers it
Genie is the interface. It resolves questions against the gold schema, generates and runs SQL, and returns results with the query visible. Certified metric definitions are pinned in the space so the same question always resolves the same way, and multi-turn context lets users drill down conversationally.

What we learned
Gold-layer quality matters more than prompt tuning — narrowing the space to purpose-built marts fixed most wrong answers. Business vocabulary has to be defined explicitly, since "delinquent" and "at risk" mean different things to different teams. Financial metrics must be precomputed, not derived on the fly, so results reconcile with official reporting. And a benchmark question set with known answers is what turns "seems good" into a measurable accuracy number.

money Performance and cutomer Insights

amitsharma20 — Tue, 25 Aug 2026 19:32:03 GMT

Announcement | How Databricks Genie Code Automated 90% of Data Ingestion for a Major Railroad

Tushar_Parekar — Thu, 20 Aug 2026 14:37:49 GMT

Databricks is highlighting how a leading Canadian transportation and logistics company used Genie Code, Unity Catalog, custom Agent Skills, and Databricks Apps to modernize legacy data pipelines at scale. The approach automated more than 90% of new table ingestion and reduced pipeline delivery from days to minutes.

What’s new

Generate pipelines from a short prompt: A compact YAML request can produce production-ready ingestion artifacts, including table definitions, historical loads, streaming ingestion, incremental merges, and tests.
Grounded in live metadata: Genie Code uses Unity Catalog to inspect source and target schemas, match columns, identify transformations, and validate required fields before generating code.
Enterprise standards built in: A custom Agent Skill packages the company’s naming conventions, audit fields, deduplication rules, merge logic, soft-delete handling, and testing patterns so they can be reused consistently.
Human review where it matters: A Databricks App helps data designers review source-to-target mappings and transformation logic before code generation, keeping business expertise in the workflow.
Deterministic, governed outputs: Genie Code handles discovery and orchestration, while rule-based templates generate repeatable PySpark and Spark SQL artifacts designed to run through Lakeflow Jobs.

👉 Read the full post here

Genie Ontology Benchmark

Mailendiran — Thu, 20 Aug 2026 10:05:02 GMT

Hi Everyone,

Now that Genie ontology is in preview, has anyone done benchmark like with & without genie ontology ?

If yes, share your experience .

If no, please suggest an idea for doing the benchmark.

Solution: Simplify Genie Agent Instruction Updates Across Multiple Spaces

harisrinivasay — Tue, 11 Aug 2026 05:54:18 GMT

Dear All,

As the number of Genie Agents grows, maintaining consistent instructions across multiple Genie Spaces becomes increasingly cumbersome. Updating the same instructions manually in each space requires significant effort and is both time-consuming and error-prone.

To simplify this process, we have created a notebook-based solution that automatically updates Genie instructions across multiple spaces. To use it, simply replace the following values in the notebook:

Genie Space IDs
Databricks Workspace URL
Personal Access Token (PAT)
Desired Instructions

Once configured, the notebook will automatically apply the same instructions to all specified Genie Spaces, eliminating the need for repetitive manual updates.

This should significantly reduce maintenance effort and ensure consistency across your Genie Agents.

Attached notebook code:

================================start here====================================

import requests
import json

workspace_url = "<your databricks workspace url"
token = "<your PAT token"

headers = {
"Authorization": f"Bearer {token}",
"Content-Type": "application/json"
}

space_ids = [
"<genie space id 1>",
"<genie space id 2>",
"<genie space id n>
]

NEW_INSTRUCTION = """
Your Role: Databricks SQL Analyst.
Rule 1: Use only tables, columns, joins, and logic defined in this Genie space. Never invent any.
Rule 2: Before generating a query, validate all rules.
<Your extended prompt Rules>
"""

for space_id in space_ids:
try:
# Get existing space
url = f"{workspace_url}/api/2.0/genie/spaces/{space_id}"
response = requests.get(
f"{url}?include_serialized_space=true",
headers=headers
)

response.raise_for_status()

space = response.json()

serialized = json.loads(space["serialized_space"])

# Replace instructions
serialized["instructions"]["text_instructions"] = [{
"content": [NEW_INSTRUCTION]
}]

payload = {
"serialized_space": json.dumps(serialized)
}

update_resp = requests.patch(
url,
headers=headers,
json=payload
)

print(
f"{space_id}: {update_resp.status_code}"
)
except requests.exceptions.RequestException as e:
print(f"{space_id}: Request failed - {e}")
except Exception as e:
print(f"{space_id}: Unexpected error - {e}")

=========================================end here================================