- 234 Views
- 4 replies
- 3 kudos
Mongo Db connector - Connection timeout when trying to connect to AWS Document DB
I am on Databricks Run Time LTE 14.3 Spark 3.5.0 Scala 2.12 and mongodb-spark-connector_2.12:10.2.0. Trying to connect to Document DB using the connector and all I get is a connection timeout. I tried using PyMongo, which works as expected and I can ...
- 234 Views
- 4 replies
- 3 kudos
- 3 kudos
Hi @rijin-thomas - Can you please allow the CIDR block for databricks account VPC from aws document db sg ( Executor connectivity stated by@bianca_unifeye ) .
- 3 kudos
- 42 Views
- 1 replies
- 0 kudos
Tab navigation between fields in dashboards is random
Tab navigation between fields in published dashboards seem very random.I have a dashboard with multiple text input fields (mapped to query paramters / filters). I expect to move logically between them when pressing tab (keyboard navigation), but I mo...
- 42 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @tvdh! You can share this as product feedback so it’s visible to the Databricks product team and can be tracked and prioritized.
- 0 kudos
- 2996 Views
- 2 replies
- 1 kudos
Resolved! Databricks APP OBO User Authorization
Hi All,We are using on-behalf of user authorization method for our app and the x-forwarded-access-token is expiring after sometime and we have to redeploy our app to rectify the issue. I am not sure what is the issue or how we can keep the token aliv...
- 2996 Views
- 2 replies
- 1 kudos
- 1 kudos
I am confronted with a similar error. I am also using obo user auth and have implemented accessing the token via st.context.headers.get('x-forwarded-access-token') for every query and do not save it in a cache. Still, after 1 hour, i am hit with the...
- 1 kudos
- 11 Views
- 0 replies
- 0 kudos
DLT Continuous Pipeline load
Hi All,In our project we are working on the DLT pipeline with the DLT tables as target running in continuous mode.These tables are common for multiple countries, and we go live in batches for different countries.So, every time a new change is request...
- 11 Views
- 0 replies
- 0 kudos
- 183 Views
- 5 replies
- 1 kudos
databricks all-purpose cluster
getting below error-Failure starting repl. Try detaching and re-attaching the notebook. while executing notebook and can see cluster have all installed lib.
- 183 Views
- 5 replies
- 1 kudos
- 1 kudos
Hi,we are not using hive metastore anywhere not sure why that host ((host=consolidated-westeuropec2-prod-metastore-0.mysql.database.azure.com)(port=3306))is coming in driver log ,will i need to do whitelist for that .we are having other use case simi...
- 1 kudos
- 51 Views
- 1 replies
- 0 kudos
Merge with schema evolution fails because of upper case columns
The following is a minimal reproducible example of what I'm facing right now.%sql CREATE OR REPLACE TABLE edw_nprd_aen.bronze.test_table ( id INT ); INSERT INTO edw_nprd_aen.bronze.test_table VALUES (1); SELECT * FROM edw_nprd_aen.bronze.test_tab...
- 51 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Dhruv-22,It's actually not a bug. Let me explain what's happening.The Root CauseThe issue stems from how schema evolution works with Delta Lake's MERGE statement, combined with Spark SQL's case-insensitivity settings.Here's the key insight: spark...
- 0 kudos
- 90 Views
- 2 replies
- 2 kudos
Resolved! [PATH_NOT_FOUND] Structured Streaming uses wrong checkpoint location
I'm trying to perform a simple example using structured streaming on a directory created as a Volume. The use case is purely educational; I am investigating various forms of triggers. Basic info:Catalog: "dev_catalog"Schema: "stream"Volume: "streamin...
- 90 Views
- 2 replies
- 2 kudos
- 2 kudos
Your checkpoint code looks correct. What is the source of `df`? Is it `/Volumes/dev_catalog/default/streaming_basics/` ? The path looks incorrect - add `stream` to it.
- 2 kudos
- 54 Views
- 0 replies
- 0 kudos
How to use/install a driver in Spark Declarative Pipelines (ETL)?
Salutations,I'm using SDP for an ETL that extracts data from HANA and put it in the Unity Catalog. I defined a Policy with the needed driver:But I get this error:An error occurred while calling o1013.load. : java.lang.ClassNotFoundException: com.sap....
- 54 Views
- 0 replies
- 0 kudos
- 26 Views
- 0 replies
- 0 kudos
DLT Gold aggregation with apply_change
I am building a Gold table using Delta Live Tables (DLT). The Gold table contains aggregated data derived from a Silver table. Aggregation happens monthly. However, the requirement is Only the current (year, month) should be recalculated. Previous mo...
- 26 Views
- 0 replies
- 0 kudos
- 33 Views
- 0 replies
- 0 kudos
Lakebridge SetUp Issue
Hi,I'm getting the below error upon executing databricks labs lakebridge analyze command. All the dependencies have been installed before execution of the command. Can someone please give a solution, or suggest if anything is missing? Below attached ...
- 33 Views
- 0 replies
- 0 kudos
- 28 Views
- 0 replies
- 0 kudos
Databricks App deployment fails: mysqlclient build error (pkg-config not found)
Hi Community Members,I’m deploying a Python project as a Databricks App, but deployment fails during dependency installation with: ERROR: Failed to build 'MySQL client' pkg-config: not found Exception: Can not find valid pkg-config nameThe dependenc...
- 28 Views
- 0 replies
- 0 kudos
- 30 Views
- 0 replies
- 0 kudos
Databricks App deployment failing due to mysqlclient build error (pkg-config not found)
Hi Team,I am trying to deploy a Python project as a Databricks App, but the deployment fails during dependency installation with the following error: ERROR: Failed to build 'mysqlclient' when getting requirements to build wheel pkg-config: not found ...
- 30 Views
- 0 replies
- 0 kudos
- 44 Views
- 0 replies
- 0 kudos
All purpose cluster, SQL Warehouse and Job Cluster are not executing the code
All purpose cluster, SQL Warehouse and Job Cluster are not executing the spark code in Pro and Classic mode. When switched to Serverless mode they are able to execute the code. When checked with Networking team there were no subnet changes recently. ...
- 44 Views
- 0 replies
- 0 kudos
- 50 Views
- 1 replies
- 0 kudos
Zerobus Connector Issue
I was trying to implement the example posted on the below link for Zerobus connector to test its functionality on my free edition workspace but unfortunately I am getting below error.Reference Code: https://learn.microsoft.com/en-us/azure/databricks/...
- 50 Views
- 1 replies
- 0 kudos
- 0 kudos
Hey @HarishKumarM , I did some digging and found some helpful information to help you troubleshoot. What the error means Your workspace isn’t currently enrolled in the Zerobus Ingest preview. Even though Zerobus is labeled a Public Preview, it’s st...
- 0 kudos
- 59 Views
- 4 replies
- 0 kudos
checkpoint changes not working on my databricks job
Hi,I do have a job processing kafka stream using kafka.readstream process and due to some issue we changed the checkpoint path to other path and it pulled all the records and later when i changed to the original checkpoint location it is not pulling ...
- 59 Views
- 4 replies
- 0 kudos
- 0 kudos
When you swapped back to the old checkpoint, were any records flowing through, and were batches completing? It's possible that you've accumulated a big backlog with the old checkpoint, and/or records in Kafka have expired. And the "startingOffsets" o...
- 0 kudos
| User | Count |
|---|---|
| 1637 | |
| 791 | |
| 518 | |
| 349 | |
| 287 |
