- 198 Views
- 1 replies
- 0 kudos
Power BI Refresh Fails with "Unable to Continue Fetch After Reconnect" from Azure Databricks
Hi everyone,I'm refreshing a report with dataset (~10 million rows) from Azure Databricks in power Bi service. The refresh runs for about an hour and then fails with: Unable to continue fetch after reconnect. Retry limit exceeded. I am currently unab...
- 198 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi Raj,Query execution on Databricks completed in under a second but the failure is occurring during the client-side result streaming phase between Power BI and Azure Databricks.When importing Power BI streams result chunks over HTTP connections. If ...
- 0 kudos
- 326 Views
- 2 replies
- 2 kudos
Serverless Scala JAR: foreachBatch fails with RST_STREAM PROTOCOL_ERROR
Hi everyone,We're migrating a Scala Structured Streaming application (Kinesis → Delta) from classic compute to Databricks Serverless Jobs for cost optimization. During the migration, we're consistently seeing what appears to be a Spark Connect / gRPC...
- 326 Views
- 2 replies
- 2 kudos
- 2 kudos
On (1): the docs don't gate Scala foreachBatch on serverless. Limitations with Databricks Connect for Scala lists streaming foreachBatch as unavailable only on Databricks Runtime 13.3 LTS and below, and serverless limitations names only Trigger.Proce...
- 2 kudos
- 4444 Views
- 9 replies
- 4 kudos
Oracle Essbase connectivity
Team, I wanted to understand the best way of connecting to Oracle Essbase to ingest data into the delta lake
- 4444 Views
- 9 replies
- 4 kudos
- 4 kudos
It depends on your Essbase setup. If it supports REST APIs or ODBC/JDBC, those are usually the best options for loading data into a Delta Lake. Share your Essbase version and target platform (e.g., Microsoft Fabric or Databricks) for more specific gu...
- 4 kudos
- 6689 Views
- 6 replies
- 8 kudos
[UNRESOLVED_ROUTINE] Cannot resolve function `date_format`
Hi all,We are getting the following error log in a Workflow:AnalysisException: [UNRESOLVED_ROUTINE] Cannot resolve function `date_format` on search path [`system`.`builtin`, `system`.`session`]. SQLSTATE: 42883The Workflow consists in different noteb...
- 6689 Views
- 6 replies
- 8 kudos
- 8 kudos
Having a similar issue when moving from DBR 13 to DBR 16.4 in a multi-task Job. The first task works ok but the 2nd with identical code but different params fails. For me it fails with Cannot resolve function `and` (I assume from the .between(from_da...
- 8 kudos
- 881 Views
- 4 replies
- 7 kudos
Genie Ontology
One question that came to mind regarding Genie Ontology is its level of automated curation. From what I've seen, Databricks appears to be taking a different approach from many vendors, where ontologies are often manually designed.Are there meaningful...
- 881 Views
- 4 replies
- 7 kudos
- 7 kudos
Today, Genie Ontology provides per-answer citations, but not yet a full reviewable changelog or diff of newly learned context. For regulated environments, that means customers can see what informed an answer, but not yet review every underlying ontol...
- 7 kudos
- 397 Views
- 1 replies
- 1 kudos
Trunked-based development model for Databricks question
Hi there,My name is William Do (a senior engineer at Hub24, Australia). I am writing to seek for your advice on a Trunk-based development model that we are hoping to implement for our Databricks project.Problem statementTo give you some context, I am...
- 397 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi William,The release train in your GitFlow workflow model is what stops you iterating, and long-running feature branches are what jam up a naive trunk-based model.Where I would gently push back is on the proposed hybrid. Reintroducing a develop bra...
- 1 kudos
- 579 Views
- 3 replies
- 5 kudos
AnalysisException: [UNRESOLVED_ROUTINE] Cannot resolve routine `=`
Hi all,We are experiencing an AnalysisException when migrating our Databricks cluster runtime from 13.3 LTS to 16.4 LTS (Scala 2.13), and would appreciate your help in identifying the root cause.**Error:**```AnalysisException: [UNRESOLVED_ROUTINE] Ca...
- 579 Views
- 3 replies
- 5 kudos
- 5 kudos
Having a similar issue when moving from DBR 13 to DBR 16.4 in a multi-task Job. The first task works ok but the 2nd with identical code but different params fails. For me it fails with Cannot resolve function `and` (I assume from the .between(from_da...
- 5 kudos
- 4732 Views
- 10 replies
- 3 kudos
Resolved! Unable to create secret scope -"Fetch request failed due expired user session"
Hi everyone,I’m trying to create an Azure Key Vault-backed secret scope in a Databricks Premium workspace, but I keep getting this error: Fetch request failed due expired user sessionSetup details:Databricks workspace: PremiumAzure Key Vault: Owner p...
- 4732 Views
- 10 replies
- 3 kudos
- 3 kudos
Hi @lofidev I resolved by requesting a policy exemption window to create scopes. The issue was due to a restrictive policy. Precisely what @lingareddy_Alva summarized earlier on this thread "When Databricks tries to programmatically grant its SP Get/...
- 3 kudos
- 2505 Views
- 3 replies
- 1 kudos
Databricks Vs Fabric use case
Hi Team,We've noticed that for some use cases, customers are proposing a architecture with A) Fabric in the Gold layer and reporting in Azure Power BI, while using Databricks for the Bronze and Silver layers. However, we can also have the B) Gold lay...
- 2505 Views
- 3 replies
- 1 kudos
- 1 kudos
When looking at a data fabric vs data lake strategy, the data lake approach shines when you need a high-performance landing zone for vast amounts of structured, semi-structured, and unstructured data. It gives data science and engineering teams tota...
- 1 kudos
- 1621 Views
- 4 replies
- 1 kudos
Resolved! Design Oracle Fusion SCM to Azure Databricks
Hello Techie,I am planning to migrate All module of Oracle fusion scm data to Azure Databricks.Do we have only option of BICC (Business Intelligence Cloud Connector), OR any other option avaialble.Can anyone please help me with reference architecture...
- 1621 Views
- 4 replies
- 1 kudos
- 1 kudos
BICC is the standard choice for bulk Oracle Fusion SCM extraction, but REST APIs and OIC are also good options depending on your use case. Your architecture and data freshness requirements will determine the best approach.
- 1 kudos
- 824 Views
- 1 replies
- 1 kudos
R plots are not rendering properly again
It looks like we have another workspace-side notebook UI regression for R plots (something similar happened last May).This time, R plots are displaying but have several very strange symptoms related to scaling:1. Larger figures (that are too big to b...
- 824 Views
- 1 replies
- 1 kudos
- 1 kudos
My Notebook, which previously worked fine, is now scaling all plots to `repr.plot.width` by `repr.plot.width`.
- 1 kudos
- 13075 Views
- 11 replies
- 9 kudos
Resolved! What are powerfull data quality tools/libraries to build data quality framework in Databricks ?
Dear Community Experts,I need your expert advice and suggestions on development of data quality framework. What are powerfull data quality tools or libraries are good to go for development of data quality framework in Databricks ? Please guide team.R...
- 13075 Views
- 11 replies
- 9 kudos
- 9 kudos
I have used Databricks labs - DQX for a failrly complex data quality implementation and it handlled the mix of built in validations and custom business rules well. One thing we learned is that performance depended much more on the underlying data lay...
- 9 kudos
- 471 Views
- 4 replies
- 2 kudos
Auto-Termination Did Not Trigger on Production Cluster Despite 20-Minute Inactivity
Hello everyone,I recently encountered an issue with one of our production all-purpose clusters where the auto-termination feature did not work as expected.The cluster was configured with "Terminate after 20 minutes of inactivity", but it never termin...
- 471 Views
- 4 replies
- 2 kudos
- 2 kudos
@van45678 -- @balajij8 's diagnosis matches what your Spark UI shows, so I will just add three things from the operational side, one for the postmortem and two for making sure the next occurrence costs you hours instead of five days.(1) Driver logs a...
- 2 kudos
- 215 Views
- 1 replies
- 0 kudos
Installing App from Marketplace
Hi All,I'm using the free edition and I'm trying to install a free app from the Marketplace. After the installation, it's asking me to provide secrets.Could you please guide me on where I can obtain these secrets? Do I need to get them from the app o...
- 215 Views
- 1 replies
- 0 kudos
- 0 kudos
Good question, and the answer is usually simpler than it looks: those secrets are almost always YOUR OWN credentials for whatever third-party service the app talks to — not something the developer sends you.Here's the model: Marketplace apps aren't a...
- 0 kudos
- 422 Views
- 3 replies
- 1 kudos
(Design Decision) Why VOID in STRUCT is not castable to VARIANT?
I asked a question recently about VOID in STRUCT failing to cast to VARIANT, whereas a VOID column is castable to VARIANT. LinkEarlier I thought it might be an error since cast was possible somehow. But the community members replied that this is the ...
- 422 Views
- 3 replies
- 1 kudos
- 1 kudos
I ran some tests.**Answer — verified at source level and across six runs, Spark 3.5.2 → 4.2.0(evidence dossier attached):**There are two independent rejections here sharing one error class, which iswhat made this confusing:1. `CAST(struct AS VARIANT)...
- 1 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 582 | |
| 349 | |
| 287 |
