cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Lakehouse Architecture


Forum Posts

kaihuGFS
by New Contributor II
  • 275 Views
  • 3 replies
  • 1 kudos

Question regarding permission of azure databricks account console

We are facing an issue while onboarding pipelines to Azure Databricks.Our Databricks environment is deployed on an Azure subscription where all of us have owner permissions. However, we're encountering problems when trying to access the Azure Databri...

  • 275 Views
  • 3 replies
  • 1 kudos
Latest Reply
MaximeGendre
New Contributor III
  • 1 kudos

Hello, Databricks admin account console is not linked to your Azure subscriptions.If you are owner of your sub, you will be admin of the workspaces instanciated in the sub. That's all.You need to find the identified Databricks admin for your company ...

  • 1 kudos
2 More Replies
Vadimalk
by New Contributor II
  • 119 Views
  • 0 replies
  • 0 kudos

Windows ODBC connection error

Hi all,I'm just started learning Databricks and have created a community-level workspace and loaded few tables.Now I'm trying to get access to the data from Excel ODBC connector following the guide here:https://docs.databricks.com/en/integrations/exc...

  • 119 Views
  • 0 replies
  • 0 kudos
singhals
by New Contributor II
  • 406 Views
  • 1 replies
  • 1 kudos

Spark event log frequently missing rollover logs

Hi,We are currently using the event log to gain some visibility to what spark is doing under the hood but we frequently find that rollover logs are missing and we are left with a partial set. Is there a reason for this or a bug being tracked? Is ther...

  • 406 Views
  • 1 replies
  • 1 kudos
Latest Reply
alpacas
New Contributor II
  • 1 kudos

I'm also having this issue  , it seems to occur much more frequently on azure than aws, but even on aws jobs sometimes some of the logs will be missing

  • 1 kudos
dbrixr
by New Contributor II
  • 612 Views
  • 2 replies
  • 0 kudos

Reverse colors in dark mode

It seems that Databricks implements its dark mode by applying the invert filter so that all colors are reversed. This is problematic if one wants to create some sort of html widget or rich output since this filter is passed down to the result of disp...

  • 612 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anushree_Tatode
  • 0 kudos

Hi @dbrixr   

  • 0 kudos
1 More Replies
rdvc
by New Contributor
  • 227 Views
  • 1 replies
  • 0 kudos

Features tab under Machine learning shows error

Cannot resolve metastore id, go back. FeatureStoreError- Workspace Feature Store has been deprecated in the current workspace. Databricks recommends using Feature Engineering in Unity Catalog. Please reach out to system admin if you have any more que...

  • 227 Views
  • 1 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 0 kudos

Hi @rdvc,Could it be that this one workspace that has this problem is not configured to use Unity Catalog? 

  • 0 kudos
unalluri9999
by New Contributor
  • 301 Views
  • 1 replies
  • 0 kudos

How to access smtplib with ACL enables cluster

I am having a tough time to connect to smtp server from databricks with ACL enabled cluster , I am able to access with out ACL enabled cluster what can be done to access it via ACL enabled cluster import smtplibfrom email.mime.multipart import MIMEMu...

  • 301 Views
  • 1 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 0 kudos

Hi @unalluri9999 ,What error do you get? By ACL you mean IP access lists or something different? Your workspace is vnet injected? 

  • 0 kudos
thibault
by Contributor II
  • 253 Views
  • 0 replies
  • 0 kudos

Streaming job update

Hi! Using bundles, I want to update a running streaming job. All good until the new job gets deployed, but then the job needs to be stopped manually so that the new assets are used and it has to be started manually. This might lead to the job running...

  • 253 Views
  • 0 replies
  • 0 kudos
DrK
by New Contributor III
  • 1168 Views
  • 4 replies
  • 0 kudos

Open Delta Sharing and Deletion Vectors

Hi,Just experimenting with open delta sharing and running into a few technical traps.  Mainly that if deletion vectors are enabled on a delta table (which they are by default now) we get errors when trying to query a table (specifically with PowerBI)...

  • 1168 Views
  • 4 replies
  • 0 kudos
Latest Reply
DarioBarbadillo
New Contributor II
  • 0 kudos

I am also experiencing this issue when connecting through Power BI connector. @DrK did you manage to solve it?

  • 0 kudos
3 More Replies
jlagos
by New Contributor II
  • 251 Views
  • 0 replies
  • 0 kudos

Survey about cluster configuration and optimization

Hi everyone, I hope you are very well.In the research for my university we are conducting a survey to Databricks users, aimed to gather information about how cluster configuration and optimization is faced in the industry.If you like you can answer i...

  • 251 Views
  • 0 replies
  • 0 kudos
Maxi1693
by New Contributor II
  • 2818 Views
  • 2 replies
  • 0 kudos

Error running 80 task at same time in Job, how limit this?

Hi! I have a Job running to process multiple streaming tables. In the beginning, it was working fine, but now I have 80 tables running in this job, the problem is that all the runs are trying to run at the same time throwing an error. Is there a way ...

  • 2818 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 0 kudos

Hi @Maxi1693, It appears that you’re encountering issues with parallel execution of tasks in your Databricks job. Let’s address this by considering a few strategies: Concurrency Limit for Tasks: Databricks allows a maximum of 1000 concurrent tas...

  • 0 kudos
1 More Replies
achistef
by New Contributor III
  • 1169 Views
  • 5 replies
  • 6 kudos

Resolved! Secret scope with Azure RBAC

Hello!We have lots of Azure keyvaults that we use in our Azure Databricks workspaces. We have created secret scopes that are backed by the keyvaults. Azure supports two ways of authenticating to keyvaults:- Access policies, which has been marked as l...

  • 1169 Views
  • 5 replies
  • 6 kudos
Latest Reply
achistef
New Contributor III
  • 6 kudos

That is very helpful, thank you for your answers.FYI there is some outdated documentation about this topic https://learn.microsoft.com/en-us/azure/databricks/security/secrets/secret-scopes#configure-your-azure-key-vault-instance-for-azure-databricks

  • 6 kudos
4 More Replies
jlowe
by New Contributor
  • 704 Views
  • 1 replies
  • 0 kudos

DABs vs API

DABs seem like a great way to build a project based workflow.  However, I have a need to take a user's SQL and create a TABLE/VIEW based on it.  We plan to use the API to submit the request but is this the best way going forward?  Should we consider ...

  • 704 Views
  • 1 replies
  • 0 kudos
Latest Reply
Shazaamzaa
New Contributor III
  • 0 kudos

DABs use Terraform which in turn calls the API anyway. So, I guess it comes down to personal preference of deployment consistency, code maintenance etc. However, from your description of "taking a user's SQL" I'm assuming some sort of real-time appli...

  • 0 kudos
Meghana_Vasavad
by New Contributor III
  • 307 Views
  • 1 replies
  • 0 kudos

Assistance Required for Enabling Unity Catalog in Databricks Workspace

Hi,I hope this message finds you well.I am reaching out regarding a concern with Databricks Administrator privileges. I have an Azure subscription and I use Azure Databricks for my tutorials, but I currently do not have Global Administrator access, w...

  • 307 Views
  • 1 replies
  • 0 kudos
Latest Reply
Walter_C
Honored Contributor
  • 0 kudos

Hello, each account should have at least one account administrator that is the one that has the permissions on the managed account and that can give you this access if applicable or assist you with the set up of the UC, you might need to ask internal...

  • 0 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels