- 2785 Views
- 6 replies
- 0 kudos
Unable to Create Secret Scope in Databricks – “Fetch request failed due to expired user session”
I’m trying to create an Azure Key Vault-backed Secret Scope in Databricks, but when I click Create, I get this error:Fetch request failed due to expired user sessionI’ve already verified my login, permissions. I also tried refreshing and re-signing i...
- 2785 Views
- 6 replies
- 0 kudos
- 0 kudos
Hi @AnandGNR Here is the youtube link refer this https://www.youtube.com/watch?v=6HQCZNW7XwY&t=800s
- 0 kudos
- 348 Views
- 3 replies
- 0 kudos
databricks-connect serverless GRPC issue
Queries executed via Databricks Connect v17 (Spark Connect / gRPC) onserverless compute COMPLETE SUCCESSFULLY on the server side (Spark tasksfinish, results are produced), but the Spark Connect gRPC channel FAILSTO DELIVER results back to the client ...
- 348 Views
- 3 replies
- 0 kudos
- 0 kudos
This is a well-known class of issue with gRPC/HTTP2 long-lived streams being killed by network intermediaries. The fact that the Databricks SQL Connector (poll-based HTTP/1.1) works perfectly while Spark Connect (gRPC/HTTP2 streaming) fails is the ke...
- 0 kudos
- 811 Views
- 1 replies
- 1 kudos
Resolved! Accessing Azure Databricks Workspace via Private Endpoint and On-Premises Proxy
Public access to the Azure Databricks workspace is currently disabled. Access is required through a Private Link (private endpoint – api_ui).A private endpoint has already been configured successfully:Virtual Network: Vnet-PE-ENDPOINTSubnet: Snet-PE-...
- 811 Views
- 1 replies
- 1 kudos
- 1 kudos
This is a classic hub-spoke + on-premises hybrid networking scenario. Here's how to architect it end-to-end. Architecture Overview The traffic flow will be: VM (VNet-App) --> ExpressRoute/VPN Gateway --> On-Prem Proxy Server --> ExpressRoute/VPN Gate...
- 1 kudos
- 517 Views
- 2 replies
- 2 kudos
Resolved! DELTA Merge taking too much Time
Hi Legends, I have a timeseries DELTA table having 707.1GiB, 7702 files, 262 Billion rows. (Mainly its timeseries data). This table is clustered on 2 columns (Timestamp col & 2nd one is descriptive column)I have designed a pipeline which runs every w...
- 517 Views
- 2 replies
- 2 kudos
- 2 kudos
Great question -- slow MERGE is one of the most common Delta Lake performance issues. Here's a systematic checklist: 1. Partition Pruning in the MERGE Condition The #1 cause of slow MERGEs is missing the partition column in your ON clause. If your ta...
- 2 kudos
- 521 Views
- 3 replies
- 0 kudos
Resolved! Invoking one job from another to execute a specific task
I have multiple tasks, each working with different tables. Each table has dependencies across Bronze, Silver, and Gold layers. I want to trigger and run a specific task independently, instead of running all tasks in the job. How can I do this? Also, ...
- 521 Views
- 3 replies
- 0 kudos
- 0 kudos
1. Go to job and left click on task you want to run .2. Click on play button(highlighted in yellow in attachment )3. This make sure that you run only 1 task at a time and not the whole job .
- 0 kudos
- 898 Views
- 7 replies
- 6 kudos
Resolved! Primary key constraint not working
I've created a Lakeflow job to run 5 notebook tasks, one for each silver table- Customers, Accounts, Transactions, Loans and Branches.In Customers notebook, after writing the data to delta table using auto loader, I'm applying the non null and primar...
- 898 Views
- 7 replies
- 6 kudos
- 6 kudos
@AanchalSoni Capturing the columns as Primary key helps users and tools understand relationships in the data. You can create Primary Key with RELY for optimization in some cases by skipping redundant operations.Distinct EliminationWhen you apply a DI...
- 6 kudos
- 929 Views
- 7 replies
- 2 kudos
Unable to create secret scope -"Fetch request failed due expired user session"
Hi everyone,I’m trying to create an Azure Key Vault-backed secret scope in a Databricks Premium workspace, but I keep getting this error: Fetch request failed due expired user sessionSetup details:Databricks workspace: PremiumAzure Key Vault: Owner p...
- 929 Views
- 7 replies
- 2 kudos
- 2 kudos
Hi @AnandGNR ,Try to do following. Go to your KeyVault, then in Firewalls and virtual networks set:"Allow trusted Microsoft services to bypass this firewall."
- 2 kudos
- 570 Views
- 2 replies
- 8 kudos
Congratulations to Matei Zaharia - CTO Databricks on the ACM Prize in Computing
When I saw the news that Matei Zaharia received the 2025 ACM Prize in Computing, I felt genuinely happy. It was not just another award announcement. It felt like a proud moment for the whole data engineering community. His work has helped shape the w...
- 570 Views
- 2 replies
- 8 kudos
- 8 kudos
@Brahmareddy, what a beautiful tribute! It’s so inspiring to hear how that meeting at the Summit stayed with you.We’re so lucky to have contributors like you who recognize the heart behind the tech. Cheers to Matei and the whole Databricks family!
- 8 kudos
- 130 Views
- 1 replies
- 1 kudos
Why Data Privacy Matters More Than Ever in the Digital Age
In today’s hyper-connected world, Data Privacy has become a critical concern for individuals and businesses alike. Every time we browse a website, use an app, or make an online purchase, we leave behind a trail of personal information. This data can ...
- 130 Views
- 1 replies
- 1 kudos
- 1 kudos
Totally agreed there @romquesta, really nice summarization!Did you check the Project Glasswing?
- 1 kudos
- 868 Views
- 6 replies
- 4 kudos
Best Practices for Implementing Automated, Scalable, and Auditable Purge Mechanism on Azure Databric
Hi All, I'm looking to implement an automated, scalable, and auditable purge mechanism on Azure Databricks to manage data retention, deletion and archival policies across our Unity Catalog-governed Delta tables.I've come across various approaches, s...
- 868 Views
- 6 replies
- 4 kudos
- 4 kudos
Here is my action plan if it helps! Phase 1: Foundation ☐ Migrate to UC managed tables (if not already) ☐ Enable Predictive Optimization at catalog level ☐ Set delta.deletedFileRetentionDuration per layer Phase 2: Retention Policies ☐ Enab...
- 4 kudos
- 137 Views
- 1 replies
- 0 kudos
Does anyone know the Databricks-specific Python syntax highlight rules?
The documentation on databricks.com says the following, in the context of configuring Python linting via pyproject.toml:You can also disable Databricks-written syntax highlighting rules with a block such as:[tool.databricks]disabled_rules = ['DB01', ...
- 137 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi — you're right that these Databricks-specific rule codes (DB01, DB03, etc.) are not documented anywhere publicly. The notebook editor docs only mention them as a configuration example without explaining what each rule checks. What We Know The DB* ...
- 0 kudos
- 267 Views
- 1 replies
- 1 kudos
variant_explode_outer stop working after the last DBX runtime patch
Hi All,I import following JSON to delta table into VARIANT column:{ "data": [ { "group": 1, "manager": "no", "firstname": "John", "lastname": "Smith", "active": "false", ...
- 267 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi, I've been testing this on a workspace at my end and see exactly the same thing. I'd first recommend raising a support ticket for this. In the meantime you can use the following workaround: I reproduced it on DBR 18.0 using readStream + cloudFil...
- 1 kudos
- 267 Views
- 2 replies
- 0 kudos
Databricks workflows for APIs with different frequencies (cluster keeps restarting)
Title: Databricks workflows for APIs with different frequencies (cluster keeps restarting)Hey everyone,I’m stuck with a Databricks workflow design and could use some advice.Currently, we are calling 70+ APIs Right now the workflow looks something l...
- 267 Views
- 2 replies
- 0 kudos
- 0 kudos
You're right that job clusters are the wrong fit here. The cold start time (including serverless, which is still 25-50s) makes anything under 5 minutes impractical when the cluster terminates between runs. The simplest approach: all-purpose cluster +...
- 0 kudos
- 600 Views
- 2 replies
- 0 kudos
Resolved! Run failed with error message Cluster was terminated. Reason: JOB_FINISHED (SUCCESS)
I am running a notebook through workflow using all purpose cluster("data_security_mode": "USER_ISOLATION"). I am seeing some strange behaviour with the cluster during the run. While the job is still running cluster gets terminated with the Reason: Re...
- 600 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi — the JOB_FINISHED (SUCCESS) termination reason is the key clue here. It means another job that was using the same all-purpose cluster finished, and its completion triggered the cluster termination — taking your still-running job down with it. Mos...
- 0 kudos
- 435 Views
- 2 replies
- 1 kudos
Resolved! Drill-down support in Databricks SQL (Lakeview) Dashboards
Hi All,Does Databricks SQL (Lakeview) Dashboards support native drill-down functionality (for example: Category → Subcategory → SKU)?Currently, we see support for cross-filtering, parameters, and drill-through within the same dataset, but hierarchica...
- 435 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi — good question. You're right that Lakeview doesn't have native hierarchical drill-down (click Category → auto-expand to Subcategory → SKU). But you can get fairly close by combining the features you mentioned. Here are the practical patterns: 1. ...
- 1 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 561 | |
| 349 | |
| 287 |
