<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic Re: Databricks notebook Issue in Data Engineering</title>
    <link>https://community.databricks.com/t5/data-engineering/databricks-notebook-issue/m-p/144641#M52362</link>
    <description>&lt;P&gt;Hi, there a few things that could cause these types of problems.&lt;/P&gt;
&lt;P&gt;1. Azure service availablity (when these happen check the Azure service availability to make sure there are no outages)&lt;/P&gt;
&lt;P&gt;2. Local network connection problems (verify all your other internet connections are working fine)&lt;/P&gt;
&lt;P&gt;3. The biggest and most likely cause is that the clusters your trying to run on are incorrectly sized for the jobs, so they are slow or you're unable to connect at all.&lt;/P&gt;
&lt;P&gt;I would recommend starting by looking at the cluster utilisation metrics in the UI, you should be able to see whether the clusters you're trying to run are over utilised. Look for the following&amp;nbsp;&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;UL&gt;
&lt;LI class="p8i6j0a"&gt;&lt;STRONG&gt;CPU Usage:&lt;/STRONG&gt; Consistently high (&amp;gt;80%) usage may indicate under-provisioned nodes.&lt;/LI&gt;
&lt;LI class="p8i6j0a"&gt;&lt;STRONG&gt;Memory Usage:&lt;/STRONG&gt; Look for memory saturation, which can lead to task spill or slowdowns.&lt;/LI&gt;
&lt;LI class="p8i6j0a"&gt;&lt;STRONG&gt;Disk I/O:&lt;/STRONG&gt; High or maxed-out disk I/O could be a bottleneck if your tasks read/write intensive datasets.&lt;/LI&gt;
&lt;/UL&gt;
&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="emma_s_0-1768932414168.png" style="width: 400px;"&gt;&lt;img src="https://community.databricks.com/t5/image/serverpage/image-id/23143i3023140914DC958C/image-size/medium?v=v2&amp;amp;px=400" role="button" title="emma_s_0-1768932414168.png" alt="emma_s_0-1768932414168.png" /&gt;&lt;/span&gt;&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
    <pubDate>Tue, 20 Jan 2026 18:07:04 GMT</pubDate>
    <dc:creator>emma_s</dc:creator>
    <dc:date>2026-01-20T18:07:04Z</dc:date>
    <item>
      <title>Databricks notebook Issue</title>
      <link>https://community.databricks.com/t5/data-engineering/databricks-notebook-issue/m-p/144147#M52271</link>
      <description>&lt;P&gt;Hello Databricks Community,&lt;/P&gt;&lt;P&gt;I’m facing multiple issues while working in Azure Databricks notebooks, and I’d appreciate guidance or troubleshooting suggestions.&lt;/P&gt;&lt;H3&gt;Issue 1: Failed to reconnect&lt;/H3&gt;&lt;P&gt;While running a notebook, I frequently see a “Failed to reconnect” message in the UI.&lt;/P&gt;&lt;P&gt;Error details:&lt;/P&gt;&lt;DIV class=""&gt;&lt;DIV class=""&gt;org.apache.http.conn.HttpHostConnectException: &lt;SPAN class=""&gt;&lt;SPAN class=""&gt;Connect &lt;SPAN class=""&gt;to westus.azuredatabricks.net:&lt;SPAN class=""&gt;443 [&lt;SPAN class=""&gt;10.115&lt;SPAN class=""&gt;.69&lt;SPAN class=""&gt;.106] failed: Connection timed &lt;SPAN class=""&gt;out (Connection timed &lt;SPAN class=""&gt;out)&lt;/SPAN&gt;&lt;/SPAN&gt;&lt;/SPAN&gt;&lt;/SPAN&gt;&lt;/SPAN&gt;&lt;/SPAN&gt;&lt;/SPAN&gt;&lt;/SPAN&gt;&lt;/SPAN&gt;&lt;P&gt;This causes the notebook to stay in a &lt;EM&gt;Waiting state and prevents execution.&lt;/EM&gt;&lt;/P&gt;&lt;H3&gt;Issue 2: Network connection timeout&lt;/H3&gt;&lt;P&gt;The error suggests a timeout while connecting to the Databricks control plane (westus.azuredatabricks.net:443).&lt;BR /&gt;This occurs intermittently and impacts productivity.&lt;/P&gt;&lt;H3&gt;Issue 3: Notebooks taking longer than expected to run&lt;/H3&gt;&lt;P&gt;Even when the notebook starts successfully:&lt;/P&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;Jobs take significantly longer than expected&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Some cells remain in Waiting state for a long time&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Performance degradation is noticeable compared to earlier runs&lt;/P&gt;&lt;H3&gt;Environment details:&lt;/H3&gt;&lt;UL&gt;&lt;LI&gt;&lt;P&gt;Azure Databricks (Region: West US)&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Notebook language: Python (PySpark)&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Workload includes reading/transformation and writing data to Snowflake&lt;/P&gt;&lt;/LI&gt;&lt;LI&gt;&lt;P&gt;Issue observed across multiple notebooks&lt;/P&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;/LI&gt;&lt;/UL&gt;&lt;/DIV&gt;&lt;/DIV&gt;</description>
      <pubDate>Thu, 15 Jan 2026 11:26:36 GMT</pubDate>
      <guid>https://community.databricks.com/t5/data-engineering/databricks-notebook-issue/m-p/144147#M52271</guid>
      <dc:creator>DataGuy2</dc:creator>
      <dc:date>2026-01-15T11:26:36Z</dc:date>
    </item>
    <item>
      <title>Re: Databricks notebook Issue</title>
      <link>https://community.databricks.com/t5/data-engineering/databricks-notebook-issue/m-p/144641#M52362</link>
      <description>&lt;P&gt;Hi, there a few things that could cause these types of problems.&lt;/P&gt;
&lt;P&gt;1. Azure service availablity (when these happen check the Azure service availability to make sure there are no outages)&lt;/P&gt;
&lt;P&gt;2. Local network connection problems (verify all your other internet connections are working fine)&lt;/P&gt;
&lt;P&gt;3. The biggest and most likely cause is that the clusters your trying to run on are incorrectly sized for the jobs, so they are slow or you're unable to connect at all.&lt;/P&gt;
&lt;P&gt;I would recommend starting by looking at the cluster utilisation metrics in the UI, you should be able to see whether the clusters you're trying to run are over utilised. Look for the following&amp;nbsp;&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;UL&gt;
&lt;LI class="p8i6j0a"&gt;&lt;STRONG&gt;CPU Usage:&lt;/STRONG&gt; Consistently high (&amp;gt;80%) usage may indicate under-provisioned nodes.&lt;/LI&gt;
&lt;LI class="p8i6j0a"&gt;&lt;STRONG&gt;Memory Usage:&lt;/STRONG&gt; Look for memory saturation, which can lead to task spill or slowdowns.&lt;/LI&gt;
&lt;LI class="p8i6j0a"&gt;&lt;STRONG&gt;Disk I/O:&lt;/STRONG&gt; High or maxed-out disk I/O could be a bottleneck if your tasks read/write intensive datasets.&lt;/LI&gt;
&lt;/UL&gt;
&lt;P&gt;&lt;span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="emma_s_0-1768932414168.png" style="width: 400px;"&gt;&lt;img src="https://community.databricks.com/t5/image/serverpage/image-id/23143i3023140914DC958C/image-size/medium?v=v2&amp;amp;px=400" role="button" title="emma_s_0-1768932414168.png" alt="emma_s_0-1768932414168.png" /&gt;&lt;/span&gt;&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;
&lt;P&gt;&amp;nbsp;&lt;/P&gt;</description>
      <pubDate>Tue, 20 Jan 2026 18:07:04 GMT</pubDate>
      <guid>https://community.databricks.com/t5/data-engineering/databricks-notebook-issue/m-p/144641#M52362</guid>
      <dc:creator>emma_s</dc:creator>
      <dc:date>2026-01-20T18:07:04Z</dc:date>
    </item>
  </channel>
</rss>

