- 2377 Views
- 2 replies
- 0 kudos
Zip archive with PowerShell "Error: The zip file may not be valid or may be an unsupported version."
Zip archive "Error: The zip file may not be valid or may be an unsupported version."We are trying to upload a ZIP archive to a Databricks workspace for faster and atomic uploads of artifacts. The expected behaviour is that we can run the following co...
- 2377 Views
- 2 replies
- 0 kudos
- 0 kudos
The error message "Error: The zip file may not be valid or may be an unsupported version" when importing a zip archive via the Databricks CLI is a known issue, especially with zip files created using PowerShell's Compress-Archive or [System.IO.Compre...
- 0 kudos
- 2317 Views
- 3 replies
- 1 kudos
Install python packages on serverless compute in DLT pipelines (using asset bundles)
Has anyone figured out how to install packages on serverless compute using asset bundle,s similar to how we handle it for jobs or job tasks?I didn’t see any direct option for this, apart from installing packages manually within a notebook.I tried ins...
- 2317 Views
- 3 replies
- 1 kudos
- 1 kudos
Installing Python packages on Databricks serverless compute via asset bundles is possible, but there are some unique limitations and required configuration adjustments compared to traditional jobs or job tasks. The core methods to install packages fo...
- 1 kudos
- 1911 Views
- 1 replies
- 0 kudos
Implementing ADB Autoloader with Managed File Notification Mode for UC Ext Location (public preview)
Hi everyone,I'm planning to implement Azure Databricks Auto Loader using the Databricks-managed file notification mode for an external location registered in Unity Catalog. I understand this feature is currently in public preview, and I’d love to hea...
- 1911 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, Azure Databricks Auto Loader with Databricks-managed file notification mode for external locations in Unity Catalog has been successfully implemented by users, especially since it entered public preview in 2025, and it's designed to make file di...
- 0 kudos
- 2177 Views
- 3 replies
- 1 kudos
DABs, policies and cluster pools
My scenario,A policy called 'Job Pool', which has the following overrides:"instance_pool_id": { "type": "unlimited", "hidden": true }, "driver_instance_pool_id": { "type": "unlimited", "hidden": true }I have an asset bundle that sets a new cluster as...
- 2177 Views
- 3 replies
- 1 kudos
- 1 kudos
You are experiencing validation errors assigning a driver to an on-demand pool and workers to a spot pool in your Databricks Asset Bundle (DAB) configuration because the 'spot_bid_max_price' attribute is being forced by policies—even when the pools a...
- 1 kudos
- 1694 Views
- 1 replies
- 0 kudos
Simba Spark Driver fails for big datasets in Excel
Hello, I am getting the following error when I want to extract data from Databricks via VBA code. The code for the connection is:Option ExplicitConst adStateClosed = 0Public CnAdo As New ADODB.ConnectionDim DSN_name As StringDim WB As WorkbookDim das...
- 1694 Views
- 1 replies
- 0 kudos
- 0 kudos
The code you provided for connecting to Databricks via VBA appears structurally sound, but the cause of the error you are experiencing could stem from several typical issues encountered when using ADODB with Databricks ODBC connections from Excel VBA...
- 0 kudos
- 1772 Views
- 2 replies
- 1 kudos
Resolved! Doubt with range_join hints optimization, using INSERT INTO REPLACE WHERE
HelloIm optmizing a big notebook and have encountered many times the tip from databricks that says "Unused range join hints". Reading the documentation for reference, I have been able to supress that warning in almost all cells, but some of then rema...
- 1772 Views
- 2 replies
- 1 kudos
- 1 kudos
There is no official documentation covering the use of range_join hints directly with the INSERT INTO ... REPLACE WHERE operation in Databricks—existing documentation around range joins focuses only on explicit joining operations, not on conditional ...
- 1 kudos
- 1912 Views
- 1 replies
- 2 kudos
Update for databricks-dlt pip package
Hello, With the recent changes to Delta Live Tables, I was wondering when the python stub will be updated to reflect the new methods that are available ?Link to the Pypi repo:databricks-dlt·PyPI
- 1912 Views
- 1 replies
- 2 kudos
- 2 kudos
The Python stub for Delta Live Tables (DLT), which helps with local development by providing API specs, docstring references, and type hints, is available as the databricks-dlt package on PyPI. However, this library only provides interfaces to the DL...
- 2 kudos
- 75 Views
- 1 replies
- 0 kudos
Network error on subsequent runs using serverless compute in DLT
Hello,When running on a serverless cluster in DLT our notebook first tries to install some python whls onto the cluster. We have noticed that when in development and running a pipeline many times over in a short space of time between runs that the pi...
- 75 Views
- 1 replies
- 0 kudos
- 0 kudos
The error you’re seeing (“Network is unreachable” repeated during pip installs) on a DLT (Delta Live Table) serverless cluster, especially after the first successful run, is a common issue that appears to affect Databricks pipelines run repeatedly on...
- 0 kudos
- 84 Views
- 2 replies
- 0 kudos
Resolved! Databricks Workflow
I have a query. I have multiple job (workflow)present in my workspace. Those job runs regularly. Multiple task present in those jobs. Few task having notebook that contain for each code in it. now when a job runs that particular task execute the for ...
- 84 Views
- 2 replies
- 0 kudos
- 0 kudos
To programmatically capture iteration-level information for tasks running inside a Databricks Workflow Job that uses the "for each" loop construct, you will primarily rely on the Databricks Jobs REST API (v2.1) and possibly the Databricks Python SDK....
- 0 kudos
- 75 Views
- 1 replies
- 1 kudos
VS Code Python file execution
Hi Everyone,I'm using the Databricks VS Code Extension to develop and deploy Asset Bundles. Usually we work with Notebooks and use the "Run File as Workflow" function. Now I'm trying to use pure python file for a new use case and tried to use the "Up...
- 75 Views
- 1 replies
- 1 kudos
- 1 kudos
You're encountering a common issue when using the Databricks VS Code Extension's "Upload and Run File" with pure Python files, especially in a secure, VNet-injected Azure Databricks deployment. Here’s a direct summary of what’s happening and how you ...
- 1 kudos
- 56 Views
- 2 replies
- 0 kudos
%run notebook fails in Job mode with Py4JJavaError (None.get), but works in interactive notebook
Hi everyone,I’m facing an issue when executing a Databricks job where my notebook uses %run to include other notebooks. I have a final notebook added as a task in a job, and inside that notebook I use %run to call another notebook that contains all ...
- 56 Views
- 2 replies
- 0 kudos
- 0 kudos
This issue with %run in Databricks notebooks—where everything works interactively in the UI, but fails in a job context with java.util.NoSuchElementException: None.get—is a relatively common pain point for users leveraging notebook modularization. Th...
- 0 kudos
- 56 Views
- 1 replies
- 1 kudos
Migrating from on-premises HDFS to Unity Catalog - Looking for advice on on-prem options
Hi,We’re currently running a Databricks installation with an on-premises HDFS file system. As we’re looking to adopt Unity Catalog, we’ve realized that our current HDFS setup has limited support and compatibility with Unity Catalog.Our requirement: W...
- 56 Views
- 1 replies
- 1 kudos
- 1 kudos
Unity Catalog does not natively support HDFS, and its primary design assumes cloud object storage (such as S3, ADLS, or GCS) as the backing store for both managed and external tables. For organizations restricted to on-premises storage, the situation...
- 1 kudos
- 61 Views
- 2 replies
- 0 kudos
Scheduling and triggering jobs based on time and frequency precedence
I have a table in Databricks that stores job information, including fields such as job_name, job_id, frequency, scheduled_time, and last_run_time.I want to run a query every 10 minutes that checks this table and triggers a job if the scheduled_time i...
- 61 Views
- 2 replies
- 0 kudos
- 0 kudos
You could add a job with a scheduled based trigger that runs every 10 minutes. The task at the start of the job runs a SQL query against the job information table and uses the logic you described above to output a boolean value. Then feed that boolea...
- 0 kudos
- 20 Views
- 0 replies
- 0 kudos
Time window for "All tables are updated" option in job Table Update Trigger
I've been using the Table Update Trigger for some SQL alert workflows. I have a job that uses 3 tables with an "All tables updated" trigger:Table 1 was updated at 07:20 UTCTable 2 was updated at 16:48 UTCTable 3 was updated at 16:50 UTC-> Job is trig...
- 20 Views
- 0 replies
- 0 kudos
- 60 Views
- 1 replies
- 1 kudos
Broken s3 file paths in File Notifications for auto loader
Suddenly at "2025-10-23T14:12:48.409+00:00", coming file paths from file notification queue started to be urlencoded. Hence, our pipeline gets file not found exception. I think something has changed suddenly and broke notification system. Here are th...
- 60 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello @aonurdemir, Could you please re-run your pipeline now and check? This issue should be mitigated now. It is due to a recent internal bug that led to the unexpected handling of file paths with special characters. You should set ignoreMissingFile...
- 1 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 482 | |
| 349 | |
| 287 |
