All Generative AI posts

Re: Databricks Knowledge Assistant

IM_01 — Fri, 14 Aug 2026 19:26:44 GMT

@suryaprayaga , I see the instructed-retriever-1 generates pair of query & filter, there can be filters that vector-index does not contain, what all metadata fields are persisted on vector-index , in case of llama-index we define the metadata columns to be persisted in vector-index right so how it works in case of KA. Wanted to understand the crux behind it.

Re: facing issue in llm models

Sumit_7 — Fri, 14 Aug 2026 07:53:57 GMT

@shrm80 Following could be the reason:
- Verification delays, usually takes sometime ~ 24hrs
- Check AWS marketplace in Billing
- If all good, raise a support ticket

In the meantime, you could test it with open-weights models (Llama).

Re: How to use AI for photo filter during registration like OwnMates?

Sumit_7 — Fri, 14 Aug 2026 07:47:38 GMT

@C_45 Great, add more references to the project.

How to use AI for photo filter during registration like OwnMates?

C_45 — Thu, 13 Aug 2026 22:25:31 GMT

I am creating a website where user can upload the pic but it will be automatically filter if the picture created by AI.

Some social website are using it

#ai #artificial

facing issue in llm models

shrm80 — Thu, 13 Aug 2026 18:03:22 GMT

i have signed up to databricks using aws but when i try to access model i m getting error Error (403): PERMISSION_DENIED: The endpoint is temporarily disabled due to a Databricks-set rate limit of 0.

Re: Testing ai_parse_document vs PyMuPDF for PDF extraction

LucasFazzi — Thu, 13 Aug 2026 14:03:39 GMT

Great points here! I'm using a similar hybrid approach in production:

PyMuPDF runs on everything first, it extracts native text from digital PDFs (fast, free, zero AI cost). Any PDF that returns no text gets tagged as needs_ocr.

Only the needs_ocr documents go through ai_parse_document, scanned PDFs, complex layouts, etc. This keeps AI costs minimal.

For the consumption layer, instead of ai_query we chunk the extracted text (4K chars, 200 overlap) and expose it through a Gold view joined with business context. Attached that to a Genie space, users can now search inside 150K+ documents in natural language and it works surprisingly well.

ai_parse_document is powerful but expensive at scale. Using it only as a fallback for what PyMuPDF can't handle gives you the best of both worlds.

Re: Building an Agentic HR Front Door on Databricks

kartikchoudhary — Mon, 10 Aug 2026 06:58:04 GMT

Interesting to see the shift from AI assistants that answer questions to agents that can actually complete tasks. For enterprise adoption, I’m curious how teams are thinking about measuring business impact here — is the focus more on reducing manual HR workflows, improving employee experience, or increasing the accuracy and speed of decisions?

Re: Databricks Knowledge Assistant

suryaprayaga — Sat, 08 Aug 2026 08:01:37 GMT

Yes this is very much implemented, however I haven't tested it much as much I have tested in Grafana's Assistant. The new feature Instructed-Retriever-1 is now conveniently called as Knowledge Assistant.

How is Instructed-Retriever different from the latest Instructed-Retriever-1 a.k.a Knowledge Assistant?

Instructed-Retriever is more linear in nature - as in, it does not have any parallel searches, query comparisons etc., However the new Knowledge Assistant can run parallel queries and intelligently combined the evidence output to rank the results to give the best evidence.

So in terms of quality and model accuracy I would say the new Knowledge Assistant is far better than the older one. However it will be proved eventually on several groundtruth facts validated earlier.

Databricks Knowledge Assistant

IM_01 — Sat, 08 Aug 2026 06:50:08 GMT

Hi All,

I've been exploring Knowledge Assistant and trying to understand how it works under the hood. I noticed it's built on the Instructed Retriever architecture, and I recently came across an update called Instructed-Retriever-1, which is supposed to deliver much faster results. Has this already been rolled out to all Databricks accounts, or is it still limited to certain users? Also, curious to understand - what exactly is different about Instructed-Retriever-1 compared to the original Instructed Retriever?

Re: The Open-Weight Revolution: A Game Changer for Our LLM Cost Optimization Odyssey

DoTA — Sat, 08 Aug 2026 02:30:48 GMT

Great analysis — this mirrors almost exactly what we've been studying in production at a regulated financial institution in Southeast Asia (banking sector).

The "token price is a poor indicator of actual costs" point is the core insight most teams miss. What you're observing with GLM-5.2 is what happens when a model's reasoning architecture aligns well with the task structure: it hits a sweet spot where fewer tokens are needed precisely because it reasons more directly. The cost-efficiency relationship is nonlinear — a model that is 3x cheaper per token but 2x more verbose per task saves you nothing; GLM-5.2 wins on both dimensions simultaneously, which is rare.

A few angles worth adding to your framework:

1. Self-hosting changes the break-even math at scale

At 10K+ tasks/day, the API pricing story shifts materially. A GLM-5.2 deployment on 8×H100s runs roughly $55–70K/month depending on cloud and reservation type. At your $1,280/day API rate scaled to 10K tasks, that is ~$38,400/month — so self-hosting breaks even around month 6, after which marginal inference cost drops 70–80%. For regulated industries with data residency requirements, the compliance value is additive on top of that.

The practical catch: cold-start latency for a 753B MoE model is non-trivial. Plan for warm routing — keep a persistent serving instance for daytime load, use spot capacity for batch overnight jobs.

2. Model routing on Databricks — practical implementation

The tiered routing you describe (Flash → GLM-5.2 → Claude) maps cleanly to Databricks AI Gateway with custom routing logic:

Host GLM-5.2 via Mosaic AI Model Serving (or proxy to external GLM API endpoint)
Add a lightweight classifier (a 1B BERT-class model is sufficient) as the routing layer — predicts task complexity from prompt features like length, entity density, and instruction type
Route to Claude only when classifier confidence + task-type signal crosses a threshold

We are seeing 60–70% of enterprise AI tasks land in the "standard" tier in practice. The 15–20% hardest tasks — multi-repository dependency resolution, architecture decisions, ambiguous multi-step reasoning — are where frontier models still earn their premium.

3. The regulated-industry sovereignty argument is underweighted

Your sovereignty point deserves more emphasis. For banking and financial services it is not just about data residency — it is about auditability. MIT-licensed open weights let you reproduce any inference exactly, version model checkpoints alongside your model risk frameworks, and demonstrate to regulators precisely what the model saw and how it responded. That is structurally impossible with black-box API calls where model versions drift silently.

One open question for anyone running this in production: how are you handling session-level cost attribution across business units? When multiple teams share a single serving endpoint, allocating inference costs back to cost centers is where we see the most operational friction. Would be interested to hear how others are solving this.

Re: How do I query genie space's actual conversation messages? (Genie space API)

empire_labs — Fri, 07 Aug 2026 23:51:02 GMT

Hi Cobs, I hit the same wall building on the Genie API. The key is that content in the message object is the user's message, not Genie's answer. The actual response lives in the attachments array, and there are several attachment types:

- attachments[].query - the SQL query Genie generated (what you're seeing now)
- attachments[].text - Genie's natural-language response, including the final summary when available, this is what the UI renders as the answer
- attachments[].suggested_questions follow-up suggestions
- attachments[].viz - visualization (beta, when enabled)

So for a message like your example, you'd pull attachments[].text (and fall back to attachments[].query when text is absent, e.g. SQL-only responses). For messages where Genie didn't run a query at all, the text attachment is the only place the answer exists.

Two practical tips:
1. Poll status until it's COMPLETED before reading attachments, the text attachment isn't guaranteed to be present while Genie is still working (and watch for error.type on failure).
2. To render a full conversation like the UI, use the List messages endpoint (GET /api/2.0/genie/spaces/{space_id}/conversations/{conversation_id}/messages) and iterate: user question from content, assistant answer from attachments[].text / attachments[].query.

Dump the full JSON of a known-good message to see all attachment types in your workspace - the docs list them but seeing the real shape helps.
Hope that unblocks you.

Re: Building an Agentic HR Front Door on Databricks

Brahmareddy — Fri, 07 Aug 2026 16:17:31 GMT

Thanks for your feedback! @wiliam65jenny

Re: Building an Agentic HR Front Door on Databricks

wiliam65jenny — Fri, 07 Aug 2026 05:25:50 GMT

Hello,

This is a great exploration of where enterprise AI is heading. The focus on moving beyond chat-based assistance toward governed action-taking agents is especially important. I like the emphasis on ontology, memory, tools, and auditability because trust will be the biggest factor in enterprise adoption.

Best Regards

Building an Agentic HR Front Door on Databricks

Brahmareddy — Fri, 07 Aug 2026 03:04:57 GMT

I have been exploring one question for the last few months:

What happens when enterprise AI moves from answering questions to actually completing tasks?

That led me to build DwaraOne, an agentic HR front door using Databricks free edition.

Most HR assistants today can explain a policy or answer a question.

But an employee usually wants more than an answer.

For example, when someone asks for leave, the agent should be able to check the balance, understand team availability, identify possible conflicts, reference the right policy, submit the request, route it to the correct approver, and track what happens next.

That is the shift I wanted to explore.

I built the current preview using Databricks Free Edition, which made the exercise even more interesting.

The architecture includes:

Unity Catalog for governed data access

Silver and Gold layers with Employee 360 views

SQL Warehouse and Delta for analytics and application data

Genie Spaces for employee, manager, and leader experiences

Lakebase for conversation and workflow state

Governed agent tools for actions

Memory across conversations

Audit logging for agent decisions and actions

One of my biggest learnings was that agent architecture should not start with the chatbot.

It should start with the data model, permissions, context, tools, and auditability.

For me, the important loop became:

Ontology → Memory → Tools → Audit

An agent that can act should also be explainable and accountable.

Another interesting lesson was that Free Edition did not feel like a limitation.

It actually pushed me to think more carefully about governance, semantic design, lineage, and architecture before adding more complexity.

The current preview uses synthetic data and supports scenarios like leave requests, policy guidance, onboarding, offboarding, workforce analytics, manager insights, consent tracking, and agent health monitoring.

I am still improving it and would really value feedback from the Databricks community.

What would you add to this architecture?

And what is the first enterprise task you would trust an agent to complete?

Preview: dwaraone.online

Re: Supervisor Agent or Knowledge assistant not available

emma_s — Tue, 04 Aug 2026 15:46:04 GMT

Hi,

Sorry you're still having issues but glad it seems to be working on the AWS one. It should be working in that region in Azure as far as I can tell. It could be an issue with model availability on Azure, at the point you tried it. If you have an Azure agreement with support you can actually file a support ticket through that and they may be able to advise.

Thanks,

Emma

Re: Supervisor Agent or Knowledge assistant not available

tensorwrangler — Mon, 03 Aug 2026 13:44:47 GMT

Azure : swedencentral - everything should be supported
Databricks (native ? - the one you convert to premium after a trial, not triggered from AWS or Azure don't know how you call that deploy option) seems to be running on AWS us-west-2. Also a region where everything is supported.

| just notied that the supervisor agent started working on us-west-2. I did contact sales (As indicated in the error msg) but never got a reply back. perhaps they did something ?

the one on Azure stil shows me the same error

Re: Supervisor Agent or Knowledge assistant not available

honey_sharma — Mon, 03 Aug 2026 08:42:05 GMT

Hi, if you're seeing "Supervisor Agent or Knowledge Assistant not available" across multiple Databricks workspaces (including Databricks-native and Azure), even with preview features enabled and appropriate billing, it's worth checking whether these features are supported in your workspace region and enabled at both the account and workspace levels. In some cases, availability depends on the cloud provider, workspace region, account entitlements, or a phased rollout rather than workspace configuration. You can review the prerequisites here:

https://docs.databricks.com/aws/en/agents/

https://docs.databricks.com/gcp/en/agents/agent-bricks/multi-agent-supervisor

https://docs.databricks.com/aws/en/generative-ai/ai-assistant/

Could you also share which cloud provider (AWS, Azure, or GCP) and region your workspaces are running in? If everything appears to be configured correctly, this may require Databricks Support to verify feature availability for your account.

Re: Getting error in databricks agent response

ajaygshah — Fri, 31 Jul 2026 08:39:22 GMT

Hello! So I am using the built-in code when we initiate the databricks custom app using agent. After tweaking and using AI by Gemini and Claude, I was finally able to start the conversation and receive the response! Sharing the code below for anyone to play with. Of course, this code only returns the first round of conversation and subsequently fails in second turn. I may look into this further, but it seems I may have to flag this to Databricks support as the built-in template should work out of the box regardless it is a reasoning or an instruction model.

import json import logging from contextlib import AsyncExitStack from datetime import datetime from typing import AsyncGenerator # --- PYDANTIC MONKEY-PATCH FOR REASONING MODELS --- def _flatten_text_payload(value): if value is None: return "" if isinstance(value, str): return value if isinstance(value, list): parts = [] for chunk in value: if isinstance(chunk, dict): if chunk.get("type") in ("text", "output_text"): parts.append(str(chunk.get("text", ""))) elif "text" in chunk: parts.append(str(chunk.get("text", ""))) elif isinstance(chunk, str): parts.append(chunk) else: parts.append(str(chunk)) return "".join(parts) if isinstance(value, dict): if value.get("type") in ("text", "output_text"): return str(value.get("text", "")) return str(value.get("text", str(value))) return str(value) def _patch_pydantic_class(cls): if cls is None: return orig_init = cls.__init__ def patched_init(self, *args, **kwargs): if "text" in kwargs: kwargs["text"] = _flatten_text_payload(kwargs["text"]) orig_init(self, *args, **kwargs) cls.__init__ = patched_init orig_validate = getattr(cls, "model_validate", None) if orig_validate: @classmethod def patched_validate(target_cls, obj, *args, **kwargs): if isinstance(obj, dict) and "text" in obj: obj["text"] = _flatten_text_payload(obj["text"]) return orig_validate(obj, *args, **kwargs) cls.model_validate = patched_validate try: from openai.types.responses import ResponseOutputText as OpenAIResponseOutputText _patch_pydantic_class(OpenAIResponseOutputText) except ImportError: pass try: from mlflow.types.responses import ResponseOutputText as MLflowResponseOutputText _patch_pydantic_class(MLflowResponseOutputText) except ImportError: pass # --------------------------------------------------- import mlflow from agents import Agent, Runner, function_tool, set_default_openai_api, set_default_openai_client from agents.tracing import set_trace_processors from databricks.sdk import WorkspaceClient from databricks_openai import AsyncDatabricksOpenAI from databricks_openai.agents import McpServer from mlflow.genai.agent_server import invoke, stream from mlflow.types.responses import ( ResponsesAgentRequest, ResponsesAgentResponse, ResponsesAgentStreamEvent, ) from agent_server.utils import build_mcp_url, get_session_id logger = logging.getLogger(__name__) set_default_openai_client(AsyncDatabricksOpenAI()) set_default_openai_api("chat_completions") set_trace_processors([]) mlflow.openai.autolog() logging.getLogger("mlflow.utils.autologging_utils").setLevel(logging.ERROR) def _sanitize_item_dict(dump: dict) -> dict: """ Repair malformed 'text' leaves (some reasoning models return text as a nested list/dict instead of a plain string) WITHOUT collapsing the container shapes (content / summary) that the Agents SDK's chat_completions converter expects to remain lists of parts. IMPORTANT: do not flatten `content` or `summary` themselves into plain strings -- Converter.items_to_messages() in the openai-agents SDK indexes into these as list-of-parts (e.g. content[0]["text"]) when reconstructing multi-turn history. Flattening them causes: TypeError: string indices must be integers, not 'str' on the *second* turn, once history round-trips back through the API. """ # Fix a malformed top-level "text" leaf without touching containers if isinstance(dump.get("text"), (list, dict)): dump["text"] = _flatten_text_payload(dump["text"]) # "content" must stay a list-of-parts for the chat_completions converter -- # only repair malformed "text" leaves inside each part content = dump.get("content") if isinstance(content, list): for part in content: if isinstance(part, dict) and isinstance(part.get("text"), (list, dict)): part["text"] = _flatten_text_payload(part["text"]) elif content is not None and not isinstance(content, (str, list)): dump["content"] = _flatten_text_payload(content) # Reasoning items use "summary": [{"type": "summary_text", "text": ...}] summary = dump.get("summary") if isinstance(summary, list): for part in summary: if isinstance(part, dict) and isinstance(part.get("text"), (list, dict)): part["text"] = _flatten_text_payload(part["text"]) return dump def sanitize_input_messages(input_items): clean = [] for item in input_items: dump = item.model_dump() if hasattr(item, "model_dump") else item if isinstance(dump, dict): clean.append(_sanitize_item_dict(dump)) else: clean.append(dump) return clean @function_tool def get_current_time() -> str: """Get the current date and time.""" return datetime.now().isoformat() def create_agent(mcp_servers: list[McpServer] | None = None) -> Agent: return Agent( name="Agent", instructions="You are a helpful assistant.", model="databricks-qwen35-122b-a10b", tools=[get_current_time], mcp_servers=mcp_servers or [], ) @invoke() async def invoke_handler(request: ResponsesAgentRequest) -> ResponsesAgentResponse: if session_id := get_session_id(request): mlflow.update_current_trace(metadata={"mlflow.trace.session": session_id}) try: async with AsyncExitStack() as stack: agent = create_agent() messages = sanitize_input_messages(request.input) result = await Runner.run(agent, messages) cleaned_outputs = [] for item in result.new_items: input_item = item.to_input_item() if isinstance(input_item, dict): input_item = _sanitize_item_dict(input_item) cleaned_outputs.append(input_item) return ResponsesAgentResponse(output=cleaned_outputs) except Exception: logger.exception("invoke_handler failed") raise @stream() async def stream_handler( request: ResponsesAgentRequest, ) -> AsyncGenerator[ResponsesAgentStreamEvent, None]: if session_id := get_session_id(request): mlflow.update_current_trace(metadata={"mlflow.trace.session": session_id}) try: async with AsyncExitStack() as stack: agent = create_agent() messages = sanitize_input_messages(request.input) result = Runner.run_streamed(agent, input=messages) # Directly stream events safely without process_agent_stream_events async for event in result.stream_events(): delta_val = None if hasattr(event, "delta"): delta_val = _flatten_text_payload(event.delta) elif hasattr(event, "data") and hasattr(event.data, "delta"): delta_val = _flatten_text_payload(event.data.delta) elif hasattr(event, "item") and hasattr(event.item, "text"): delta_val = _flatten_text_payload(event.item.text) if delta_val: yield ResponsesAgentStreamEvent(delta=delta_val) except Exception: logger.exception("stream_handler failed") raise

Re: Billing structure and LLM usage costs for no-code agents

GabFernandes — Thu, 30 Jul 2026 13:04:21 GMT

Hi @r_w_ !

Estimating cost and understanding the billing model for No-Code Agents and Genie in Databricks comes down to how DBUs (Databricks Compute Units) and Token consumption are metered across three core layers:

1. Overview of Where Billing Occurs
There is no standalone "base fee" just for maintaining an Agent definition. You are billed purely based on consumption across these items:

Orchestration & Agent Execution: Charged under Serverless Real-Time Inference DBUs (Model Serving / Agent Hosting) for the compute hosting the agent logic and Supervisor routing.

SQL Warehouse Usage (for Genie Agent): When Genie generates and runs SQL queries against your Lakehouse, you are billed for the underlying Databricks SQL Warehouse DBUs (Serverless or Pro).

LLM Inference Costs: Billed via Foundation Model API (FMAPI) on a Pay-per-token basis (input/output tokens) or Provisioned Throughput if configured.

2. LLM Usage Costs (Included or Billed Separately?)
LLM costs are billed separately based on token usage, tracked via Foundation Model API / External Models Gateway.

When a Supervisor Agent or Genie routes a prompt to a managed LLM, the tokens consumed (prompt + completion) are metered through FMAPI.

The execution runtime (Serverless Inference DBUs) and the LLM token usage (FMAPI) will appear as distinct line items in your Azure/AWS/GCP Databricks bill.

3. Which LLM is Used & Can It Be Changed?
Default LLMs: By default, Genie and Agent Bricks leverage Databricks' optimized foundation models (such as DBRX, Llama 3, or partner models integrated natively).

Can you change it? Yes. You can route and customize the underlying models through the Databricks AI Gateway / Model Serving endpoints. This allows you to point your agents to different foundation models (e.g., Anthropic Claude, OpenAI GPT-4o, Llama 3, or custom fine-tuned models) depending on your accuracy and cost requirements.

Official Documentation for Cost Estimation:
Databricks Pricing Page (Model Serving & FMAPI): https://www.databricks.com/product/pricing

Databricks Genie Billing & Architecture: https://docs.databricks.com/en/genie/index.html

If my answer was helpful, please consider marking it as accepted solution!

Re: Genie task running for over 20 hours.

balajij8 — Thu, 30 Jul 2026 11:08:04 GMT

Hi egs,

You can delete the notebook or chart to get rid of the hung Genie task. Since Genie tasks are generating and executing read only exploratory queries against the SQL Warehouse, they generally don't modify the underlying Delta tables. Deleting the UI artifact must simply tear down the client session and abandon the execution state.

Before you delete it, you can first confirm whether this is just a UI desync. You can do a refresh of the workspace page or try closing and reopening the tab as it often clears a stuck visual state if the query has actually already finished or timed out on the backend.

If the task is genuinely still hanging in the UI after 20 hours, go ahead and delete the notebook or chart. To prevent it from hanging again, you can instruct Genie use a much smaller date range or add LIMIT clause in its queries.