Azure Databricks UI consuming way too much memory & laggy

[dj4]

This especially happens when the notebook is large with many cells. Even if I clear all the outputs scrolling the notebook is way too laggy. When I start running the code the memory consumption is 3-4GB minimum even if I am not displaying any data/tables. I saw a previous post about this issue couple of months ago expecting it to be fixed. But everyone in my team has this issue. As an IDE, Databricks UI is basically not usable.  Any fix?

[emma_s]

There are ongoing improvements being made to the Databricks notebook UI, and more to come to improve performance. However, you may be better to consider whether you would be better to break down the notebooks into smaller components, as browser memory is always going to struggle if too much is loaded at once. Alternatively you consider using VS code and the Databricks extension.

[dj4]

Browser memory doesnt struggle when using jupyter-lab locallly with the same exact code. Even google colab which is browser based doesnt have this issue. Also breaking down it into smaller components can be done when we're moving to production but during development you are bound to use many cells to test out things. Even with an empty notebook, the UI is not exactly smooth to work with(like jupyter-lab or colab)

[emma_s]

Hi, these are teh recommended troubleshooting steps we have:

Troubleshooting & Immediate Workarounds

1. Browser Recommendations:

   • Use an incognito/private window to avoid interference from browser extensions/ad blockers.
   • Monitor memory consumption; close other tabs and apps if running out of RAM.

2. Disable Enhanced Notebook Tabs:

   • Switch to the legacy notebook UI instead of the enhanced tabs. Go to “Settings → Developer → Tabs for notebooks and files.” This allows you to open each notebook in separate browser tabs (similar to Google Colab), reducing strain from tab switching.

3. Collect Browser Performance Data:

   • Capture a browser “performance trace” while reproducing the lag. Use developer tools (Chrome DevTools Performance tab).
   • Install third-party tools like Lag Radar to visualize client-side delay.

4. Workspace Hardware:

   • If possible, confirm hardware specifications and consider upgrading RAM/CPU if routinely working with large notebooks.

5. VDI Environment Specifics:

   • Slowdowns are frequently worse on VDIs. If possible, test locally or compare with cloud-based resources.

But really I'd recommend filing a support ticket.

 

[siva-anantha]

@dj4: Are you in a corporate proxy environment?

Databricks Browser UI uses Web Sockets and sometimes the performance issues happen due to the security checks in the traffic.