Databricks Community Genie-Powered App Challenge. Track B: Creative Thinking.
Most Genie apps answer questions. This one refuses to. You bring it a claim — "boys are better at maths", "China is the world's biggest polluter" — and Genie writes the SQL that would test it. You see the query and Genie's reasoning. The result stays sealed until you call it and stake points on your call. Then you cross-examine in plain English, Genie rewrites its own query in the same conversation, and the verdict often flips.
Links.
Code: github.com/ivanvyd/prove-it
Play with no account: prove-it.streamlit.app
Demo video:
The idea, and who it is for
People repeat numbers they cannot interrogate, and a chatbot that answers teaches deference to a new authority. So the app hands you the method instead of the answer: the query is on screen, the number is not, and you bet before you look. Built for children aged 10 to 14 in a classroom — one screen, no pupil accounts, nothing personal stored — and for anyone who has ever been handed a confident chart.
The docket
Five claims, three on real published data (Bickel, Hammel & O'Connell 1975; Our World in Data, CC BY 4.0), two on a synthetic per-pupil table.
- The average — "boys are better at maths". The trick: the hidden spread. Looks true → Busted.
- The gap that stays — "girls are better at reading". The trick: the gap that survives. Looks true → it holds.
- The paradox — "men were more likely to be admitted to Berkeley in 1973". The trick: Simpson's paradox. Looks true → Busted.
- The window — "Bulgaria halved its education spending 1991–1996". The trick: the chosen window. Looks true → Half true.
- The denominator — "China is the world's biggest polluter". The trick: the missing denominator. Looks true → Half true.
One case survives on purpose. A docket where nothing holds up would teach that everything is a trick, and cynicism is cheaper than scepticism.
One case, end to end
Berkeley, 1973. Genie's first query pools admissions by gender: men 44.5%, women 30.4%. Looks true, by fourteen points. You call it, stake it, and the rows unseal.
Then you type one sentence — break that down by department — and Genie rewrites its own query in the same conversation. Twelve rows now. Four of the six departments admitted women at a higher rate than men. Verdict: Busted. Women applied in far larger numbers to the departments that admitted almost nobody, so the pooled total followed the easy departments. That is Simpson's paradox, met in your own hands rather than in a definition.
Genie at the core
Remove Genie and no product remains — only a quiz with pre-written questions. The app depends on six Genie capabilities, each load-bearing.
- Genie writes every query. The claim goes to the Conversation API as a question in English, and the SQL on screen is what came back. The app contains no SQL at all: a build gate walks the syntax tree of every application module and fails on any SQL literal.
- Genie explains its reasoning. The query attachment carries a "thoughts" array of typed steps — understanding, data sourcing, instructions — and the app pins them beside the query as "how it got there". Most integrations throw this field away; it turned out to be the most teachable thing in the payload.
- Genie holds the rows behind a handle. A response carries an attachment id, and the rows come from a separate result call. The seal is that gap: the app shows the query and reasoning, and declines to make the second call until your call and stake are in. The sealed bag prints the real handle it is holding.
- Genie holds a conversation. Your follow-up joins the same exchange, and Genie rewrites its query with the first turn as context. That is the flip, and it is what a stateful agent does that a template cannot.
- The space's instructions are product surface. Every lesson depends on Genie answering the first question plainly — exactly as asked, volunteering nothing — so there is something left to overturn. That behaviour lives in the space instructions, versioned in the repo and pushed by script.
- Genie reports its phases. The Conversation API's statuses — fetching metadata, asking the model, executing — light a board in an "interrogation room" while you wait, so a 20-second turn reads as a scene instead of a spinner.
And you can check all of it. Each query carries Genie's own conversation and message ids, readable against the space's message history:
query v1 — conversation 01f19cef42f81e17996b65ef60c957d0
query v2 — conversation 01f19cef42f81e17996b65ef60c957d0
The same conversation, twice. The follow-up continued the exchange rather than re-asking.
What you can ask it
Everything reaches Genie in plain English, word for word. Each docket case sends its rumour as a question. Case No 0 takes yours: type any claim you have heard and Genie says whether the four tables can test it — most typed claims end in "can't tell from this data", which the app scores as a win. And the cross-examination is an editable text box: your own wording goes into the live conversation, and Genie writes the second query from it.
Architecture
Prove It (Streamlit on Databricks Apps) sends the claim, verbatim, as a question in English to the Genie Conversation API (stateful, multi-turn). The Genie space runs it through a SQL warehouse over four Unity Catalog tables: berkeley_admissions (real, 1975), country_indicators (real, OWID), emissions (real, OWID), and student_scores (synthetic, per-pupil). The app declares the Genie space as a resource in its app config and runs as its own service principal. No accounts, no persistence beyond the browser tab. The test suite asserts that no rows are fetched before a call is committed, down every path, including refusals.
What it costs to be wrong
- Called it: +100 × stake (Hunch ×1 · Fairly sure ×2 · Certain ×3)
- Verdict overturned: +250 — paid even when your call was wrong
- Case closed: +150
- Called "the data can't say", and it couldn't: +200
- Docket cleared: +500
Sure and wrong costs the multiple it would have paid. Nothing is gated, points floor at zero, and the ending reads out your calibration — Certain: 2 of 2 right — because that is a claim about you that a score cannot make.
What I learned building it
- The "thoughts" field beat the answer. Genie's typed reasoning steps teach a beginner what "interpreting a question" consists of. Render them.
- Withholding is a feature. One response, split in half: query now, rows later. That split turned a query tool into a lesson.
- Making Genie naive was harder than making it correct. The denominator case first summed a per-capita column across 270 years; the instruction fix then over-corrected and Genie volunteered per-person figures in the first answer, destroying the case. "Only when they ask" restored the clean naive draft, three probe runs out of three. Space instructions deserve the same review as code.
- Verify against what renders. A green suite and a healthy server twice hid a page that was wrong in the browser. Every case is probed against live Genie before any UI is built on it, and a test now reads this article's own figures against the data.
Try it
uv venv --python 3.12 && uv pip install -e .
PROVE_IT_OFFLINE=1 streamlit run src/prove_it/ui/app.py
Every case replays its own recorded two-turn Genie conversation, captured against a live Free Edition space with the exact wording the app sends. Setup for a real workspace is in the repo's docs/setup.md.
Play it online: prove-it.streamlit.app
