Databricks Community

haaland · Tuesday

I created a supervisor agent and asked it to plot something.
It was able to generate text and table, then output a lot of gibberish which was supposed to be its attempt of plotting.

Some of it's output (redacted):

{'text/plain': '<Figure size 1400x1000 with 2 Axes>', 'image/png': 'iVBORw0KGgoAAAANSUhEUgAABW0AAAPdCAYAAADxjUr8AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQABAABJREFUeJzs3Xd4U+UXwPFv0l3oppNZ9pIhs+wNgiiCskSmgAqi4MQF+HPi3ojIEAQFBGUIsvdGEMoeZXcCnXQlub8/Qi9NZ9ombVPO53l4aO58k5N7k5z73vNqFEVREEIIIYQQQgghhBBCCFEqaEu6AUIIIYQQQgghhBBCCCHukaStEEIIIYQQQgghhBBClCKStBVCCCGEEEIIIYQQQohSRJK2QgghhBBCCCGEEEIIUYpI

Is this a bug and will it be fixed?

anuj_lathi · yesterday

That isn't the model garbling a plot. It's a valid rich-output payload from a code execution step that your chat surface is printing as raw text instead of rendering. I can't speak to a fix timeline, but it's a rendering/handoff gap rather than a problem with your agent's logic.

What you're looking at

The dict is a Jupyter-style MIME bundle. It has a text/plain fallback (<Figure size 1400x1000 with 2 Axes>) and an image/png entry. The iVBORw0KGgo… prefix is the standard base64 header of a PNG file. So the agent ran matplotlib code successfully, and the figure came back as a base64 PNG. The response path then serialized that bundle into the message text instead of displaying it as an image.

You can confirm this by pasting the full image/png string into a notebook:

import base64
from IPython.display import Image, display

b64 = "iVBORw0KGgoAAAANSUhEUgAABW0..."  # paste the full string, not the truncated one
display(Image(data=base64.b64decode(b64)))

If the chart renders, the agent did its job and only the display layer failed.

Workarounds

Don't let the agent return the figure inline. Tell it in its instructions to save charts to a file (for example a Unity Catalog Volume path) and return the path or link, and never to print or return the figure object or its raw output. Your app or UI can then load the image from that location.
Strip or convert the bundle in your app layer. If you're calling the agent through your own code, detect image/png keys in the response, base64-decode them, and render or store the image yourself. Drop the text/plain fallback.
Use native visualization for interactive analysis. If the goal is charts over query results, display() output in a notebook or SQL editor gives you the built-in visualization editor, with aggregation, filtering, and PNG download [1]. Dashboards are another good option for anything that needs to be shared.
Inspect the trace. If you have MLflow Tracing enabled on the agent, open the trace for that run. It will show you which step produced the bundle and whether it was the tool output or the final response that carried it [2].

Will it be fixed?

I can't give you a roadmap commitment. If this is a Databricks-managed supervisor agent, inline image rendering in the response is a product behavior rather than something you can configure. Open a support ticket, or submit product feedback with the agent type, the workspace region, and a trace of the failing run. That's the fastest way to get it in front of the right team. In the meantime, option 1 or 2 gets you working charts without waiting on a change.