- 547 Views
- 1 replies
- 1 kudos
Resolved! How to stop Databricks retaining widget selection between runs?
I have a Python notebook in Databricks. Within it I have a multiselect widget, which is defined like this:widget_values = spark.sql(f''' SELECT my_column FROM my_table GROUP BY my_column ORDER BY my_column ''') widget_values = widget_values.collect(...
- 547 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello @SRJDB , What you’re running into isn’t your Python variable misbehaving—it’s the widget hanging onto its own internal state. A Databricks widget will happily keep whatever value you gave it, per user and per notebook, until you explicitly clea...
- 1 kudos
- 550 Views
- 1 replies
- 1 kudos
Resolved! DLT quota Issue
Hi Community,I am facing a weird problem within my Azure Databricks workspace, I am trying to create and run SDP, but somehow when I try to run more than 1 pipeline in parallel, it gives me an error (pasting the error message below). I currently only...
- 550 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello @AyushPaldecha09! This error usually appears due to concurrency limits. If you're already on a Premium tier, you typically shouldn’t be hitting this cap, so the best next step is to open a Databricks Support ticket and request an increase to yo...
- 1 kudos
- 1309 Views
- 4 replies
- 2 kudos
Time Travel Error when selecting from materialized view (Azure Databricks)
Hey - running into an error this morning that was brought to my attention via failed refreshes from PowerBI. We have a materialized view that, when queried with the standard pattern of 'select col1 from {schema}.table_name', returns an error of 'Cann...
- 1309 Views
- 4 replies
- 2 kudos
- 2 kudos
OUR FIXEventually we learned via DESCRIBE EXTENDED, that the table had lost its Location path in production environment. Querying via the development environment to the production catalog the location path was still visible.Solved the issue (with t...
- 2 kudos
- 969 Views
- 3 replies
- 4 kudos
Resolved! Workflow scheduling on particular working day of the month in ADB
Hi,I am looking to schedule a workflow to execute on 3rd working day. Working day here would be Mon-Fri of each month. I could not find any direct crontab solution but have created watcher file solution for it. Below code will create a watcher file a...
- 969 Views
- 3 replies
- 4 kudos
- 4 kudos
Hi @bianca_unifeye , @Poorva21 ,Thank you for your valuable responses.I have changed the main workflow trigger as file arrival in scheduling. I have scheduled the watcher file code to run daily in weekdays and as soon as the 3rd working day encounter...
- 4 kudos
- 1055 Views
- 1 replies
- 0 kudos
Resolved! Issue Genie API - different responses in UI and via API
Hi community,I created an agent with a genie tool, a wrapper around a GenieAgent connected to my Genie space (GENIE_SPACE_ID) that sends user questions and returns Genie’s textual response. I noticed I get 02 different responses when I post a questio...
- 1055 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey @D4F , What you’re seeing is normal behavior—and the good news is there are very real, very practical ways to make your Genie-based agent more consistent without resorting to a giant, brittle prompt. Let’s dig in. First, why the UI and API can r...
- 0 kudos
- 4006 Views
- 4 replies
- 3 kudos
Databricks asset bundle occasionally duplicating jobs
Since last year, we have adopted Databricks Asset Bundles for deploying our workflows to the production and staging environments. The tool has proven to be quite effective, and we currently use Azure DevOps Pipelines to automate bundle deployment, tr...
- 4006 Views
- 4 replies
- 3 kudos
- 4778 Views
- 4 replies
- 3 kudos
Delta Tables: Time-To-Live
I have seen somewhere (might have been in a Databricks Tech Talk) a Delta Table feature which allows to specify the "expiration date" of data stored in Delta Tables. Once rows surpass their time-to-live, they are automatically deleted or archived. Do...
- 4778 Views
- 4 replies
- 3 kudos
- 1510 Views
- 6 replies
- 1 kudos
Resolved! AWS_INSUFFICIENT_INSTANCE_CAPACITY_FAILURE when starting SQL Server Ingestion pipeline
Dear Community,I’m seeing a compute error when running a Databricks ingestion pipeline (Lakeflow managed ingestion) on AWS.Cloud : AWSRegion: ap−northeast−2Source: SQL Server ingestion pipelineWhen I start the ingestion pipeline, it fails with the f...
- 1510 Views
- 6 replies
- 1 kudos
- 1 kudos
Hi, I'm afraid you cannot edit compute instance type settings for SQL Server ingestion pipelines via the Databricks UI. Such changes can only be made via API.
- 1 kudos
- 1509 Views
- 6 replies
- 4 kudos
Resolved! Accessing Databricks data in Salesforce via zero copy
I have uploaded clickstream data as shown belowDo i have to mandatorily share via Delta sharing for values to be exposed in Salesforce ?At the Salesforce end i have confirmed that i have a working connector where i am able to see samples data , but u...
- 1509 Views
- 6 replies
- 4 kudos
- 4 kudos
HI Team ,Please help me my task is to connect Databrick and salesforce data cloud with zero copy . where we need databricks data in Salesforce data cloud , also just to mention my databricks workspace + ADLS stoarge is on private end point. any hel...
- 4 kudos
- 371 Views
- 1 replies
- 0 kudos
Resolved! Is there a supported method to register a custom PySpark DataSource so that it becomes visible in th
Built a custom connector using the PySpark DataSource API (DataSource V2). The connector works programmatically, but it does not appear in the Databricks Ingestion UI (Add Data → Connectors) like the Salesforce connector.Is there a supported method t...
- 371 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @venkatesh557 ,Unfortunately, the answer is no - there isn’t a supported way for you to “register” an arbitrary PySpark DataSource V2 so that it appears as a tile in the Databricks Add data → Connectors (Ingestion) UI right now
- 0 kudos
- 1255 Views
- 6 replies
- 6 kudos
Resolved! How can I pass parameters from DABs to something(like notebooks)?
I'm implementing DABs, Jobs, and Notebooks.For configure management, I set parameters on databricks.yml.but I can't get parameters on notebook after executed a job successfully. What I implemented ans Steps to the issue:Created "dev-catalog" on WEB U...
- 1255 Views
- 6 replies
- 6 kudos
- 6 kudos
Hi @tak0519 I think I found the issue! Don't worry - your DABs configuration looks correct. The problem is actually about how you're verifying the results, not the configuration itself. What's happening In your last comment, you mentioned: "Manuall...
- 6 kudos
- 599 Views
- 6 replies
- 2 kudos
Materialized view always load full table instead of incremental
My delta table are stored at HANA data lake file and I have ETL configured like below@DP.materialized_view(temporary=True) def source(): return spark.read.format("delta").load("/data/source") @dp.materialized_view def sink(): return spark.re...
- 599 Views
- 6 replies
- 2 kudos
- 2 kudos
1 more note that I'm not using Unity Catalog here, not sure if it's relevant
- 2 kudos
- 736 Views
- 3 replies
- 1 kudos
Resolved! Azure Databricks Streamlit Application - Doubts
Hi Databricks community, I am currently tasked with creating a stream lit application with the help of data bricks application feature, I have currently created a lake base instance to sync the delta table located at the unity catalog and I have also...
- 736 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @RJTECHY210 ,Yes, it's possible. You can use python sdk to achieve what you want. Here's a sample code for a reference:from databricks.sdk import WorkspaceClient from databricks.sdk.service.database import DatabaseInstance # Initialize the Worksp...
- 1 kudos
- 456 Views
- 3 replies
- 0 kudos
Unable to configure custom compute for DLT pipeline
I am trying to configure cluster for a pipeline like above, However dlt keeps using the small cluster as usual, how to resolve this?
- 456 Views
- 3 replies
- 0 kudos
- 0 kudos
i updated my CLI and deployed the job, still i dont see the clusters updates in pipeline
- 0 kudos
- 1369 Views
- 3 replies
- 3 kudos
badrequest: cannot create online table is being deprecated. creating new online table is not allowed
Hello!This seems so sudden that we cannot create online tables anymore? Is there a workaround to being able to create online tables temporarily as we need more time to move to synced tables? #online_tables
- 1369 Views
- 3 replies
- 3 kudos
- 3 kudos
Yes, the Databricks online tables (legacy) are being deprecated, and after January 15, 2026, you will no longer be able to access or create them.https://docs.databricks.com/aws/en/machine-learning/feature-store/migrate-from-online-tablesHere are few ...
- 3 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 553 | |
| 349 | |
| 287 |
