cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Lakehouse Architecture


Forum Posts

SolaireOfAstora
by New Contributor
  • 331 Views
  • 0 replies
  • 0 kudos

Databricks report error: unexpected end of stream, read 0 bytes from 4 (socket was closed by server)

Has anyone encountered this error and knows how to resolve it?"Unexpected end of stream, read 0 bytes from 4 (socket was closed by server)."This occurs in Databricks while generating reports.I've already adjusted the wait_timeout to 28,800, and both ...

  • 331 Views
  • 0 replies
  • 0 kudos
chaaaaaarlie
by New Contributor II
  • 507 Views
  • 2 replies
  • 1 kudos

Resolved! Missing 'Permissions' settings for Delta Live Tables Pipelines

Context: Azure Databricks, I am account admin, workspace admin, and pipeline owner (as confirmed via the API and visually in the Pipelines screen). When attempting to grant CAN_MANAGE access to developers for our DLT pipelines via the Databricks web ...

  • 507 Views
  • 2 replies
  • 1 kudos
Latest Reply
Walter_C
Databricks Employee
  • 1 kudos

If you click on the link of the Pipeline and then you access the kebab menu you will see the Permissions page  

  • 1 kudos
1 More Replies
sjs
by New Contributor II
  • 410 Views
  • 1 replies
  • 0 kudos

Unable to create a databricks workspace

I am unable to create a databricks workspace with vnet injection. I get this error:│ { │ "status": "Failed", │ "error": { │ "code": "InternalServerError", │ "message": "INTERNAL_ERROR: Unexpected error: Cannot call getCertifiedMetastoreFo...

  • 410 Views
  • 1 replies
  • 0 kudos
Latest Reply
sjs
New Contributor II
  • 0 kudos

The issue resolved itself when I tried to create a new resource group, dedicated to just Databricks. I don't know why that worked. If anyone know what went wrong, I would appreciate feedback!

  • 0 kudos
arkadiuszr
by New Contributor III
  • 4842 Views
  • 5 replies
  • 1 kudos

Resolved! Unable to access Live Ganglia UI

I’m having a trouble accessing Ganglia UI on a job run cluster. The job run is executed by a service principal and I would like to enable the Ganglia UI view to an user/admin group in Databricks. The error I get is: HTTP ERROR 403 PERMISSION_DENIED: ...

  • 4842 Views
  • 5 replies
  • 1 kudos
Latest Reply
shan_chandra
Databricks Employee
  • 1 kudos

Hi @arkadiuszr  -  can you please try the following steps and let us know? To resolve this error, first you want to add your service principal to the workspace: Go to admin console as a workspace adminSelect “service principal tab”Click “Add service ...

  • 1 kudos
4 More Replies
RameshRetnasamy
by Contributor
  • 9634 Views
  • 24 replies
  • 22 kudos

Resolved! Unable to login to Azure Databricks Account Console

I have a personal Azure pay-as-you-go subscription in which I have the 'Global Administrator' role. I am also the databricks account administrator.Until two weeks ago, I was able to access the databricks account console without any issues, but I am f...

Screenshot 2024-08-07 at 12.13.53.png Screenshot 2024-08-07 at 12.15.30.png Screenshot 2024-08-07 at 12.17.20.png Screenshot 2024-08-07 at 12.18.35.png
Administration & Architecture
account-console
Databricks
  • 9634 Views
  • 24 replies
  • 22 kudos
Latest Reply
champrats
New Contributor II
  • 22 kudos

Thanks, @dustinvannoy and @RameshRetnasamy!When I first started the Formula1 training course and couldn’t find the Account Console, I assumed Databricks must have changed something. Thanks for clarifying—I’m now able to follow along with the instruct...

  • 22 kudos
23 More Replies
Cstan
by New Contributor
  • 433 Views
  • 0 replies
  • 0 kudos

VPAT Form

How do I find a Voluntary Product Accessibility Template (VPAT) from Databricks?

  • 433 Views
  • 0 replies
  • 0 kudos
art1
by New Contributor III
  • 1701 Views
  • 7 replies
  • 5 kudos

Resolved! How to get rid of a pesky gen AI feature in the editor ?

Hi,The editor interface has that gen AI feature following empty lines with a cursor. I find that very distracting and irritating. More over, once a line is deleted that unsolicited thing is interfering with code (snapshots included).How to get rid of...

art1_0-1727364182823.png art1_1-1727364247306.png
  • 1701 Views
  • 7 replies
  • 5 kudos
Latest Reply
art1
New Contributor III
  • 5 kudos

Summary:I was not able to solve the UI/UX artifact on my own (on user side)The UI/UX issue was resolved somewhere on Databricks side. The UI/AX artifact is no longer interfering with work.

  • 5 kudos
6 More Replies
mannepk85
by New Contributor III
  • 308 Views
  • 1 replies
  • 0 kudos

Resolved! Attach a databricks_instance_pool to databricks_cluster_policy via terraform

Hello Team,I am trying to create a databricks instance pool and attach it to a cluster policy in our terraform code. But I am having hard time finding a good documentation. Has any one done it? Below is my sample code and I am getting errorI keep get...

  • 308 Views
  • 1 replies
  • 0 kudos
Latest Reply
mannepk85
New Contributor III
  • 0 kudos

Fixed it! "instance_pool_id" : { type = "fixed" values = "databricks_instance_pool.dev_test_cluster_pool.id"}

  • 0 kudos
IN
by New Contributor II
  • 274 Views
  • 1 replies
  • 0 kudos

Removal of account admin

Hi, I'm having issues with removing account admin (probably the first one, to which databricks account was related to). Under user management, when I hit the delete user button, it prompts:Either missing permissions to delete <user_email> or deleting...

  • 274 Views
  • 1 replies
  • 0 kudos
Latest Reply
IN
New Contributor II
  • 0 kudos

I have also tried removing account using databricks CLI: databricks account users delete <ID>Error: Either missing permissions to delete <account> or deleting own account.

  • 0 kudos
knawara
by Contributor
  • 292 Views
  • 0 replies
  • 0 kudos

Databricks Bundles - Terraform state management

Hello,I had a look at DABs today and it seems they are using Terraform under the hood. The state is stored in Databricks Workspace, in the bundle deployment directory. Is it possible to use just the state management functionality that DABs must have ...

  • 292 Views
  • 0 replies
  • 0 kudos
SanthoshKumarK
by New Contributor II
  • 560 Views
  • 2 replies
  • 0 kudos

Lakehouse federation -' on behalf of ' queries

Is it possible to achieve the following in a lake-house federation setup using Azure Databricks?1. Establish an external connection (EC1) to an external data source (EDS) using the credentials of user U1.2. Create a foreign catalog (FC1) utilizing EC...

  • 560 Views
  • 2 replies
  • 0 kudos
Latest Reply
SanthoshKumarK
New Contributor II
  • 0 kudos

Thanks for explaining the authorization flow, @rangu . In the example mentioned, does Databricks support passing the user’s credentials to an external data source? For instance, can it pass the OAuth token for the user along with the externalID crede...

  • 0 kudos
1 More Replies
TSK
by New Contributor
  • 312 Views
  • 0 replies
  • 0 kudos

GitLab on DCS, Datarbricks Container Services

I would like to set up GitLab and Grafana servers using Databricks Container Services (DCS). The reason is that our development team is small, and the management costs of using EKS are not justifiable. We want to make GitLab and Grafana accessible in...

Administration & Architecture
AWS
Container
DevOps
EKS
Kubernetes
  • 312 Views
  • 0 replies
  • 0 kudos
noorbasha534
by Contributor
  • 544 Views
  • 4 replies
  • 0 kudos

Libraries installation governance

Dear allI like to know the best practices around libraries installation on Databricks compute - all-purpose, job.The need is to screen the libraries, conduct vulnerability tests, and then let them be installed through a centralized CI/CD process. How...

  • 544 Views
  • 4 replies
  • 0 kudos
Latest Reply
noorbasha534
Contributor
  • 0 kudos

@filipniziol thanks again for your time. The thing is we like to block access to these URLs as at times we found developers & data scientists downloading packages that were marked as vulnerable by Maven.

  • 0 kudos
3 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels