cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

_sqldf not defined on Azure job cluster v12.2

piterpan
New Contributor III

Since yesterday we have errors in notebooks that were previously working.

 

 

NameError: name '_sqldf' is not defined

 

 

It was working previously.

We are on Azure databricks, usng job pool 
Driver: Standard_D4s_v5 ยท Workers: Standard_D4s_v5 ยท 1-6 workers ยท from pool ยท 12.2 LTS

We have opened a ticket on Azure, no response for now ๐Ÿ˜• 

1 ACCEPTED SOLUTION

Accepted Solutions

Tharun-Kumar
Databricks Employee
Databricks Employee

@piterpan 

This was a regression issue which impacted the jobs where _sqldf was referenced and the notebook those weren't run interactively. Our Engineering team has fixed this issue yesterday.

Could you check whether you are still facing the issue?

View solution in original post

8 REPLIES 8

Kale-S
New Contributor II

I have the same error since yesterday. 

I have rewrote all my magic SQL (%SQL) cells with the spark function:
query = """
              select * 
              from table
"""

df = spark.sql(query)

samirsaliba
New Contributor II

Same here, also on Azure Databricks. Cluster version is 12.2 LTS Photon.

ViniciusTavares
New Contributor II
Today I had the same error, the df _sqldf stopped working

aholts
New Contributor II

Having same issue since Thursday

whitebulb
New Contributor II

Same here. It was working completely fine a couple weeks ago.

Tharun-Kumar
Databricks Employee
Databricks Employee

@piterpan 

This was a regression issue which impacted the jobs where _sqldf was referenced and the notebook those weren't run interactively. Our Engineering team has fixed this issue yesterday.

Could you check whether you are still facing the issue?

Thank for your feedback.
We noticed it's working on v13.2 and we have been using v13.2 as a workaround.
I am waiting for the test on v12.2 and I'll confirm you if it's resolved for us.

piterpan
New Contributor III

I can confirm it's working again.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group