cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Lakehouse Architecture


Forum Posts

RameshSolanki
by New Contributor II
  • 250 Views
  • 1 replies
  • 0 kudos

Bring data from databricks to sharepoint list using the Power Automate

Good afternoon to all and I am new to this community.We are trying to bring data from databricks to sharepoint list using the Power Automate app (create workflow and trigger it when there is new record or exising record is modified in source table in...

  • 250 Views
  • 1 replies
  • 0 kudos
Latest Reply
RameshSolanki
New Contributor II
  • 0 kudos

Hi all, Can anyone assist me with this request ?Thanks in advance 

  • 0 kudos
JamesDryden
by New Contributor
  • 180 Views
  • 1 replies
  • 0 kudos

How are you deploying graphs?

Hi all, I have a couple of use cases that may benefit from using graphs.  I'm interested in whether anyone has graph databases in Production and, if so, whether you're using GraphFrames, Neo4j or something else?  What is the architecture you have the...

  • 180 Views
  • 1 replies
  • 0 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 0 kudos

Yes I have.I have a case where we use graphframes (using BFS).  Works fine, as long as you use the graph algorithms that are delivered with graphframes. And that can be quite limiting.We also have a case where we started with graphframes but decided ...

  • 0 kudos
vhazeleger
by New Contributor
  • 288 Views
  • 1 replies
  • 0 kudos

Tabs for notebooks

Browsing this page of the documentation, the displayed GIF shows a notebook that is opened in its own tab. I've been looking for how to enable this feature in my own workspace, but cannot find it.Does anyone know how to enable this feature?

  • 288 Views
  • 1 replies
  • 0 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 0 kudos

Nope.It seems that is some kind of new version of the UI.In the SQL editor one can open multiple tabs. But for python notebooks I have no idea.

  • 0 kudos
achistef
by New Contributor III
  • 1828 Views
  • 6 replies
  • 6 kudos

Resolved! Secret scope with Azure RBAC

Hello!We have lots of Azure keyvaults that we use in our Azure Databricks workspaces. We have created secret scopes that are backed by the keyvaults. Azure supports two ways of authenticating to keyvaults:- Access policies, which has been marked as l...

  • 1828 Views
  • 6 replies
  • 6 kudos
Latest Reply
Chamak
New Contributor II
  • 6 kudos

@daniel_sahal Where in the portal did you get the App ID for AzureDatabricks (fa5c679a-c02e-4f33-a397-7419315171b3) Application? I can't seem to find it. Thanks.

  • 6 kudos
5 More Replies
PabloCSD
by Contributor II
  • 213 Views
  • 1 replies
  • 0 kudos

How to generate an Azure Subscription from a Databricks Generated Service Principal?

Hello, I currently have a Service Principal (SP) Client_Id and its associated secret, I generated it directly from my workspace in Databricks, i was following this post: https://github.com/databricks/cli/issues/1722, but I don't know how to generate ...

  • 213 Views
  • 1 replies
  • 0 kudos
Latest Reply
VictoriaTha
New Contributor II
  • 0 kudos

Learn to summon an Azure Subscription from a Databricks-generated Service Principal. Harness the power of data with this vital step in Azure infrastructure management. Mastering it is as crucial as surviving Fnaf

  • 0 kudos
tcmx
by New Contributor II
  • 275 Views
  • 1 replies
  • 0 kudos

Resolved! Restrictions on setting environment variables in Compute Policies

As recommended by Databricks, we are trying to use Compute Policies to set environment variables, which are used by our notebooks, across clusters.However, when specifying a JSON string as env var, we are getting this error upon applying the policy t...

  • 275 Views
  • 1 replies
  • 0 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 0 kudos

This is because you use Shared access mode.This enables multiple users to use the cluster simultaneously.However, there are features that do not work on these Shared access mode clusters:https://docs.databricks.com/en/compute/access-mode-limitations....

  • 0 kudos
stevanovic
by New Contributor
  • 310 Views
  • 1 replies
  • 0 kudos

One Azure Tenant with Multiple Azure Databricks Accounts

Hi there,We have one Azure tenant with multiple subscriptions. Each subscription is a project for itself.At this moment, we have only one Azure Databricks account, and all workspaces (created under different subscriptions) are associated with it.Can ...

  • 310 Views
  • 1 replies
  • 0 kudos
Latest Reply
radothede
Contributor II
  • 0 kudos

hello @stevanovic ,as far as I understand, in Azure, you can create one databricks account per tenant, meaning for example unity catalog is also tenant-level resource.There is a fantastic blog post available here:https://community.databricks.com/t5/t...

  • 0 kudos
Kayla
by Valued Contributor
  • 283 Views
  • 3 replies
  • 2 kudos

Resolved! Silly question-Easy way to show full notebook path or owner in UI?

We have a few people working in Databricks right now in different clones of the same repository. Occasionally we'll have multiple people with the same branch open- one working, another just has it open to see what it looks like, sort of deal.This has...

  • 283 Views
  • 3 replies
  • 2 kudos
Latest Reply
radothede
Contributor II
  • 2 kudos

hi @Kayla ,I think the easiest way to check the current notebook location when opened is just hover the mouse cursor over the name of the notebook (top left, "ADE 3.1 - Streaming Deduplication" in this case) and wait for about 1-2 seconds; after that...

  • 2 kudos
2 More Replies
sharat_n
by New Contributor
  • 369 Views
  • 1 replies
  • 0 kudos

Delta Lake: Running Delete and writes concurrently

Is it safe to run a delete query when there are active writes to a delta lake table? Next question : Is it safe to run a vacuum when writes are being done actively?  

  • 369 Views
  • 1 replies
  • 0 kudos
Latest Reply
radothede
Contributor II
  • 0 kudos

Hello @sharat_n ,Yes, it is generally safe to run a DELETE query on a Delta Lake table while active writes are happening.Delta Lake is designed with ACID transactions, meaning operations like DELETE, UPDATE, and MERGE are atomic and isolated.In other...

  • 0 kudos
SolaireOfAstora
by New Contributor
  • 258 Views
  • 0 replies
  • 0 kudos

Databricks report error: unexpected end of stream, read 0 bytes from 4 (socket was closed by server)

Has anyone encountered this error and knows how to resolve it?"Unexpected end of stream, read 0 bytes from 4 (socket was closed by server)."This occurs in Databricks while generating reports.I've already adjusted the wait_timeout to 28,800, and both ...

  • 258 Views
  • 0 replies
  • 0 kudos
chaaaaaarlie
by New Contributor II
  • 379 Views
  • 2 replies
  • 1 kudos

Resolved! Missing 'Permissions' settings for Delta Live Tables Pipelines

Context: Azure Databricks, I am account admin, workspace admin, and pipeline owner (as confirmed via the API and visually in the Pipelines screen). When attempting to grant CAN_MANAGE access to developers for our DLT pipelines via the Databricks web ...

  • 379 Views
  • 2 replies
  • 1 kudos
Latest Reply
Walter_C
Databricks Employee
  • 1 kudos

If you click on the link of the Pipeline and then you access the kebab menu you will see the Permissions page  

  • 1 kudos
1 More Replies
sjs
by New Contributor II
  • 334 Views
  • 1 replies
  • 0 kudos

Unable to create a databricks workspace

I am unable to create a databricks workspace with vnet injection. I get this error:│ { │ "status": "Failed", │ "error": { │ "code": "InternalServerError", │ "message": "INTERNAL_ERROR: Unexpected error: Cannot call getCertifiedMetastoreFo...

  • 334 Views
  • 1 replies
  • 0 kudos
Latest Reply
sjs
New Contributor II
  • 0 kudos

The issue resolved itself when I tried to create a new resource group, dedicated to just Databricks. I don't know why that worked. If anyone know what went wrong, I would appreciate feedback!

  • 0 kudos
arkadiuszr
by New Contributor III
  • 4687 Views
  • 5 replies
  • 1 kudos

Resolved! Unable to access Live Ganglia UI

I’m having a trouble accessing Ganglia UI on a job run cluster. The job run is executed by a service principal and I would like to enable the Ganglia UI view to an user/admin group in Databricks. The error I get is: HTTP ERROR 403 PERMISSION_DENIED: ...

  • 4687 Views
  • 5 replies
  • 1 kudos
Latest Reply
shan_chandra
Databricks Employee
  • 1 kudos

Hi @arkadiuszr  -  can you please try the following steps and let us know? To resolve this error, first you want to add your service principal to the workspace: Go to admin console as a workspace adminSelect “service principal tab”Click “Add service ...

  • 1 kudos
4 More Replies
RameshRetnasamy
by Contributor
  • 7409 Views
  • 24 replies
  • 22 kudos

Resolved! Unable to login to Azure Databricks Account Console

I have a personal Azure pay-as-you-go subscription in which I have the 'Global Administrator' role. I am also the databricks account administrator.Until two weeks ago, I was able to access the databricks account console without any issues, but I am f...

Screenshot 2024-08-07 at 12.13.53.png Screenshot 2024-08-07 at 12.15.30.png Screenshot 2024-08-07 at 12.17.20.png Screenshot 2024-08-07 at 12.18.35.png
Administration & Architecture
account-console
Databricks
  • 7409 Views
  • 24 replies
  • 22 kudos
Latest Reply
champrats
New Contributor II
  • 22 kudos

Thanks, @dustinvannoy and @RameshRetnasamy!When I first started the Formula1 training course and couldn’t find the Account Console, I assumed Databricks must have changed something. Thanks for clarifying—I’m now able to follow along with the instruct...

  • 22 kudos
23 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels