Can anyone share experience (or point me to another reference) that describes how to configure Azure Blob storage which has NFS enabled as an external volume to Databricks ?I've succeeded in adding SMB storage to Databricks but (if I understand prope...
Apologies for the delay & many thanks for responding. Yes I've been able to mount my premium storage + NFS container as an external volume to Databricks.
Hello,I'm trying to write to Azure PostgreSQL flexible database from Azure Databricks, using PostgreSQL connector in Databricks Runtime in 12.2LTS.I'm using df.write.format("postgresql").save() to write to PostgreSQL database, but getting the follow...
Yes, this stack overflow was my reference too and adding below option made load go with no error on UUID data type in postgres columnSpoiler.option(stringtype, "unspecified").option(stringtype, "unspecified")https://stackoverflow.com/questions/409739...
Whenever I try to open my company file over a network or multi-user mode, I keep getting QB Desktop Error 6000 and something after that. The error messages on my screen vary every time I attempt to access the data file. I cannot understand the error,...
Hi, @faithlawrence98 Don’t worry; we’re here to help you.To open your company file over the network, it is required to run the QB Database Server Manager on your server computer. In case this tool isn’t running, you will most probably face QB Desktop...
I have two workflows (jobs) in data bricks (AWS) with below cluster specs (job base cluster NOT general purpose)Driver: i3.xlarge · Workers: i3.xlarge · 2-8 workers Job 1 takes 10 min to completeJob 2 takes 50 min to completeQuestions:DBU cost is sam...
Hi @Raja_fawadAhmed,
DBU Cost for Both Jobs:
Databricks pricing is based on DBUs (Databricks Units) consumed. The cost depends on the type of compute instances used and the specific workload.For your two jobs, the DBU cost would be calculated...
I am getting this error when trying to create a cluster: "Self-bootstrap failure during launch. Please try again later and contact Databricks if the problem persists. Node daemon fast failed and did not answer ping for instance"
I am looking for some help on getting databricks cluster metrics such as memory utilization, CPU utilization, memory swap utilization, free file system using REST API.I am trying it in postman using databricks token and with my Service Principal bear...
Hi Community, I am trying to understand the difference between:Delete token API: DELETE /api/2.0/token-management/tokens/{token_id}Revoke token API: POST /api/2.0/token/deleteAs, when I create more than 600 tokens - I am getting QUOTA_EXCEEDED error....
Hi,I am trying to work on address parsing and would like to use libpostal in Databricks.I have used the official python bindings: GitHub - openvenues/pypostal: Python bindings to libpostal for fast international address parsing/normalizationpip insta...
Hi @hpicatto,
The Spark Web UI provides a suite of user interfaces (UIs) for monitoring your Spark cluster. You can access it by navigating to http://<driver-node-ip>:18080 in your web browser.On the appropriate application page, click the Download ...
Hi Community, I was going through this doc: https://docs.databricks.com/api/workspace/tokens/create to and got to know, that there is a quota limit to how many token one can generate using the api: POST /api/2.0/token/create, having breached the thre...
Hi @Surajv, Let’s dive into the details of token quotas in Databricks.
Quota Limit for Token Creation:
The quota limit for creating user tokens via the API (specifically, using POST /api/2.0/token/create) is essential to manage token usage.Each u...
Hi @Surajv, When working with Databricks service principals, you can manage and view personal access tokens (PATs) associated with them.
Here’s how you can achieve this:
What is a Service Principal?
A service principal is an identity created in ...
We're looking for feedback on the Databricks free trial experience, and we need your help!
Whether you've used it for data engineering, data science, or analytics, Sujit Nair, our Product Manager on the free trial experience, and our journey archite...
Hi @Еmil, I've read through your question and believe I have a solution for you.
Here's a response to your question:
Since your job runs as a service principal, consider using OAuth M2M authentication for accessing your Azure DevOps Git repository.En...
Hi, First foray into DLT and following code exerts from the sample-DLT-notebook.I'm creating a notebook with the SQL below:CREATE STREAMING LIVE TABLE sales_orders_rawCOMMENT "The raw sales orders, ingested from /databricks-datasets."TBLPROPERTIES ...
If you change the notebook default language as opposed to using magic command. I normally have it set to Python, I've wrongly assumed DLT would transpose as can't use magic command but have to change default in order for it to work.
