Databricks Community

Dimitry

Hi guys I've added a parameter "files_mask " to a notebook, with a default value.The job running this notebook broke with error: com.databricks.dbutils_v1.InputWidgetNotDefined: No input widget named files_mask is definedCode: mask = dbutils.widgets....

Dimitry

HI allServerless V3 solved an error of mismatching python versions between driver and worker which I had on V2 (can't remember the exact wording).So I'd been running this on classic compute so far.Today I tried on serverless to a partial success - un...

Dimitry · 09-02-2025

Hi allI got a job trigger by a file event on the external location.The location and jobs triggers are working fine when uploading file via Azure Portal.I need SFTP trigger, so I went into the event grid, found subscription for the storage account on ...

Dimitry · 07-11-2025

Hi allI've followed the book for creating external location for Azure Data Lake Storage (ADLS V2) using account connectorI've granted all required permissions to the connector:I've created a "stock" container on that above mentioned "devtyremeshare" ...

Dimitry · 06-17-2025

Hi all I have 2 clusters, that look identical but one runs my UDF in parallel another one does not.The ones that do is personal, the bad one is shared.import pandas as pd from datetime import datetime from time import sleep import threading # test f...

Dimitry

Also, the same problem with SQL cells

Dimitry

I'm using Python and run it under a standard compute (not serverless).I don't have Scala cells in the notebook and it is confusing, as the other thread was about failing in Scala but working in Python.

Dimitry

I was wrong in interpreting the results. threading.get_native_id() does not work on serverless as on classic, so different threads return the same ID. The time it takes to execute the test is obviously less than 40 seconds, if it was running on a sin...

Dimitry · 09-04-2025

Yeah, but in my case I just archive the file as soon as it gets processed, and it will be handful tops. I guess majority of cases are like that and when you need more in one place... perhaps you have a budget to incorporate ADF/Function/Logic app. I ...

Dimitry · 09-04-2025

FYI - look at this Solved: Job File Event Trigger not firing for SftpCommit a... - Databricks Community - 128356You also copied the same solution. But the trick is - you DONT use events. Just turn it off. No queue.In this case File Event trigger will...

Databricks Community

User Stats

User Activity

databricks notebook parameter works in interactive mode but not in the job

Serverless - can't parallelize UDF in applyInPandas

Unreliable file events on Azure Storage (SFTP) for job trigger

Problem activating File Events for External Location / ADLS V2

Struggle to parallelize UDF

Re: databricks notebook parameter works in interactive mode but not in the job

Re: databricks notebook parameter works in interactive mode but not in the job

Re: Serverless - can't parallelize UDF in applyInPandas

Re: Unreliable file events on Azure Storage (SFTP) for job trigger

Re: Unreliable file events on Azure Storage (SFTP) for job trigger