<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:taxo="http://purl.org/rss/1.0/modules/taxonomy/" version="2.0">
  <channel>
    <title>topic SparkSession.getActiveSession() is Suddenly Broken in Python Multi-threading in Data Engineering</title>
    <link>https://community.databricks.com/t5/data-engineering/sparksession-getactivesession-is-suddenly-broken-in-python-multi/m-p/176357#M57127</link>
    <description>&lt;P&gt;Hello everyone,&lt;/P&gt;&lt;P&gt;Around 3PM EST yesterday (2026-11-12), dozens of Lakeflow jobs started failing on both classic and serverless compute with distinct symptoms&lt;/P&gt;&lt;P&gt;1. On classic compute (DBR 15.4), reading a file from S3 would result in a PATH NOT FOUND error&lt;/P&gt;&lt;P&gt;2. On serverless compute (v2 and v5), reading a file from S3 would result in a "'NoneType' has not attribute 'read'" error indicating the SparkSession object was None/null&lt;/P&gt;&lt;P&gt;The issue appears to be resolved on classic compute as of 6PM EST 2026-11-12. The issue is ongoing on serverless compute. This is the simplest bit of code that demonstrates the issue.&lt;/P&gt;&lt;LI-CODE lang="python"&gt;from concurrent.futures import ThreadPoolExecutor
from pyspark.sql import SparkSession

def test(foo):
    print(SparkSession.getActiveSession())

with ThreadPoolExecutor(max_workers=4) as executor:
    future_to_batch = {
        executor.submit(test, foo): foo for foo in [“A”, “B”, “C”, “D”]
    }&lt;/LI-CODE&gt;&lt;P&gt;Instead of a SparkSession object appearing in cell output, we see "None". Replacing SparkSession.getActiveSession() with SparkSession.builder.getOrCreate() alleviates the problem.&lt;/P&gt;&lt;P&gt;Is anyone else seeing this?&lt;/P&gt;</description>
    <pubDate>Fri, 13 Nov 2026 15:45:09 GMT</pubDate>
    <dc:creator>ifishbein</dc:creator>
    <dc:date>2026-11-13T15:45:09Z</dc:date>
    <item>
      <title>SparkSession.getActiveSession() is Suddenly Broken in Python Multi-threading</title>
      <link>https://community.databricks.com/t5/data-engineering/sparksession-getactivesession-is-suddenly-broken-in-python-multi/m-p/176357#M57127</link>
      <description>&lt;P&gt;Hello everyone,&lt;/P&gt;&lt;P&gt;Around 3PM EST yesterday (2026-11-12), dozens of Lakeflow jobs started failing on both classic and serverless compute with distinct symptoms&lt;/P&gt;&lt;P&gt;1. On classic compute (DBR 15.4), reading a file from S3 would result in a PATH NOT FOUND error&lt;/P&gt;&lt;P&gt;2. On serverless compute (v2 and v5), reading a file from S3 would result in a "'NoneType' has not attribute 'read'" error indicating the SparkSession object was None/null&lt;/P&gt;&lt;P&gt;The issue appears to be resolved on classic compute as of 6PM EST 2026-11-12. The issue is ongoing on serverless compute. This is the simplest bit of code that demonstrates the issue.&lt;/P&gt;&lt;LI-CODE lang="python"&gt;from concurrent.futures import ThreadPoolExecutor
from pyspark.sql import SparkSession

def test(foo):
    print(SparkSession.getActiveSession())

with ThreadPoolExecutor(max_workers=4) as executor:
    future_to_batch = {
        executor.submit(test, foo): foo for foo in [“A”, “B”, “C”, “D”]
    }&lt;/LI-CODE&gt;&lt;P&gt;Instead of a SparkSession object appearing in cell output, we see "None". Replacing SparkSession.getActiveSession() with SparkSession.builder.getOrCreate() alleviates the problem.&lt;/P&gt;&lt;P&gt;Is anyone else seeing this?&lt;/P&gt;</description>
      <pubDate>Fri, 13 Nov 2026 15:45:09 GMT</pubDate>
      <guid>https://community.databricks.com/t5/data-engineering/sparksession-getactivesession-is-suddenly-broken-in-python-multi/m-p/176357#M57127</guid>
      <dc:creator>ifishbein</dc:creator>
      <dc:date>2026-11-13T15:45:09Z</dc:date>
    </item>
  </channel>
</rss>

