Hey @D4F ,
What you’re seeing is normal behavior—and the good news is there are very real, very practical ways to make your Genie-based agent more consistent without resorting to a giant, brittle prompt.
Let’s dig in.
First, why the UI and API can return different answers.
Genie is nondeterministic by nature. Small differences in phrasing, context, or even ordering can lead to different outputs. That’s expected behavior. The best way to reduce variance is to keep your guidance clean and free of conflicting instructions.
From a mechanics standpoint, calling the API is effectively the same as starting a new thread in the UI—but differences in chat history, phrasing, and which context gets selected can still lead to different SQL and different answers.
Context filtering and token limits also play a big role. In long or heavily annotated spaces, important instructions or metadata can get dropped. When that happens, quality drops with it.
And finally, permissions matter. If your agent isn’t using on-behalf-of-user (OBO) authorization, it may literally be seeing a different universe of assets and metadata than the user sees in the UI.
Now, why joins and column descriptions sometimes don’t stick.
Joins have to live somewhere Genie can actually use them. That means either proper PK/FK relationships in Unity Catalog or explicit joins defined in the space’s knowledge store. If neither exists, Genie is left to guess.
Foreign or federated tables are another common pitfall. Their comments and column descriptions don’t always sync locally, which means Genie may never see them. The fix here is simple: edit the local metadata in the Genie space, or wrap the source with a materialized view that includes proper comments.
Lastly, overlong or overlapping instructions tend to get deprioritized by context filtering. When that happens, the most important joins and definitions may never make it into the model’s working context. Short, structured guidance wins here every time.
So how do you improve accuracy without writing a novel?
The strategy is curation and structure—not more prose.
Narrow the dataset aggressively. Only include the tables and columns that actually matter. Small, purposeful schemas outperform wide, noisy ones every time. Pre-join into views where it helps and hide irrelevant columns.
Invest in the knowledge store. Add clear table and column descriptions. Define synonyms. And if PK/FKs don’t exist in Unity Catalog, define JOIN relationships directly there.
Prefer example SQL over paragraphs of instruction. A few trusted example queries for common or complex questions are far more effective than generalized guidance. The same goes for defining reusable business measures as SQL expressions.
Enable data sampling and value dictionaries for key categorical fields. That gives Genie the grounding it needs to map real user language to the right columns and values.
Then iterate like an engineer. Add benchmark questions. Run tests. Use review and monitoring to spot gaps. When you see a great response, convert it into trusted instructions or canonical examples.
Hope this helps, Louis.
