Ashwin_DSA
Databricks Employee
Databricks Employee

Hi @michael365,

There is no documented hard rule that "general instructions must be 20 lines or less." Databricks guidance is to keep text instructions small, focused, and well-organised, because long or overly broad instructions can become less effective, especially over longer conversations. However, what you are seeing can happen, especially when clarification behaviour is defined only in plain-text instructions.

A few best practices that usually help:

  1. Prefer SQL expressions and example SQL over text instructions wherever possible. Those tend to be more reliable than plain text alone.
  2. Make clarification rules very explicit. Instead of a general instruction like "ask a clarification question," define the trigger condition, what detail is missing, that Genie must ask before answering and the exact clarification question it should ask.
  3. Put clarification instructions at the end of the general instructions so they are easier for Genie to prioritise.
  4. Keep the space narrowly focused and reduce ambiguity in the data model where possible, for example, by improving column descriptions, adding synonyms, or hiding confusing columns.

For your example, a stronger pattern would be something like:

"When a user asks about weight or volume, and multiple columns could match the request, ask a clarification question before generating SQL or answering. Do not assume the correct column. Ask exactly: 'Which measure do you want me to use: gross weight, net weight, or volume?' Only continue after the user selects one."

Useful references:

In short... the issue is usually less about a strict line limit, and more about instruction quality, specificity, and whether the behaviour should instead be taught through examples or SQL-based definitions.

If this answer resolves your question, could you mark it as “Accept as Solution”? That helps other users quickly find the correct fix.

Regards,
Ashwin | Delivery Solution Architect @ Databricks
Helping you build and scale the Data Intelligence Platform.
***Opinions are my own***

View solution in original post