cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Lakehouse Architecture


Forum Posts

rdvc
by New Contributor
  • 491 Views
  • 1 replies
  • 0 kudos

Features tab under Machine learning shows error

Cannot resolve metastore id, go back. FeatureStoreError- Workspace Feature Store has been deprecated in the current workspace. Databricks recommends using Feature Engineering in Unity Catalog. Please reach out to system admin if you have any more que...

  • 491 Views
  • 1 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 0 kudos

Hi @rdvc,Could it be that this one workspace that has this problem is not configured to use Unity Catalog? 

  • 0 kudos
unalluri9999
by New Contributor
  • 393 Views
  • 1 replies
  • 0 kudos

How to access smtplib with ACL enables cluster

I am having a tough time to connect to smtp server from databricks with ACL enabled cluster , I am able to access with out ACL enabled cluster what can be done to access it via ACL enabled cluster import smtplibfrom email.mime.multipart import MIMEMu...

  • 393 Views
  • 1 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 0 kudos

Hi @unalluri9999 ,What error do you get? By ACL you mean IP access lists or something different? Your workspace is vnet injected? 

  • 0 kudos
thibault
by Contributor II
  • 322 Views
  • 0 replies
  • 0 kudos

Streaming job update

Hi! Using bundles, I want to update a running streaming job. All good until the new job gets deployed, but then the job needs to be stopped manually so that the new assets are used and it has to be started manually. This might lead to the job running...

  • 322 Views
  • 0 replies
  • 0 kudos
jlagos
by New Contributor II
  • 332 Views
  • 0 replies
  • 0 kudos

Survey about cluster configuration and optimization

Hi everyone, I hope you are very well.In the research for my university we are conducting a survey to Databricks users, aimed to gather information about how cluster configuration and optimization is faced in the industry.If you like you can answer i...

  • 332 Views
  • 0 replies
  • 0 kudos
Maxi1693
by New Contributor II
  • 3063 Views
  • 1 replies
  • 0 kudos

Error running 80 task at same time in Job, how limit this?

Hi! I have a Job running to process multiple streaming tables. In the beginning, it was working fine, but now I have 80 tables running in this job, the problem is that all the runs are trying to run at the same time throwing an error. Is there a way ...

  • 3063 Views
  • 1 replies
  • 0 kudos
Latest Reply
" src="" />
This widget could not be displayed.
This widget could not be displayed.
This widget could not be displayed.
  • 0 kudos

This widget could not be displayed.
Hi! I have a Job running to process multiple streaming tables. In the beginning, it was working fine, but now I have 80 tables running in this job, the problem is that all the runs are trying to run at the same time throwing an error. Is there a way ...

This widget could not be displayed.
  • 0 kudos
This widget could not be displayed.
jlowe
by New Contributor
  • 823 Views
  • 1 replies
  • 0 kudos

DABs vs API

DABs seem like a great way to build a project based workflow.  However, I have a need to take a user's SQL and create a TABLE/VIEW based on it.  We plan to use the API to submit the request but is this the best way going forward?  Should we consider ...

  • 823 Views
  • 1 replies
  • 0 kudos
Latest Reply
Shazaamzaa
New Contributor III
  • 0 kudos

DABs use Terraform which in turn calls the API anyway. So, I guess it comes down to personal preference of deployment consistency, code maintenance etc. However, from your description of "taking a user's SQL" I'm assuming some sort of real-time appli...

  • 0 kudos
Meghana_Vasavad
by New Contributor III
  • 428 Views
  • 1 replies
  • 0 kudos

Assistance Required for Enabling Unity Catalog in Databricks Workspace

Hi,I hope this message finds you well.I am reaching out regarding a concern with Databricks Administrator privileges. I have an Azure subscription and I use Azure Databricks for my tutorials, but I currently do not have Global Administrator access, w...

  • 428 Views
  • 1 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

Hello, each account should have at least one account administrator that is the one that has the permissions on the managed account and that can give you this access if applicable or assist you with the set up of the UC, you might need to ask internal...

  • 0 kudos
wsunwall
by New Contributor II
  • 626 Views
  • 1 replies
  • 1 kudos

Disabling Default Personal Compute Policy in Terraform (AWS)

I'm looking to disable the Personal Compute Policy in Terraform, however I don't see an option to do this. According to this documentation here https://docs.databricks.com/en/admin/clusters/personal-compute.html#customize-the-personal-compute-policy ...

  • 626 Views
  • 1 replies
  • 1 kudos
Latest Reply
mggl
New Contributor II
  • 1 kudos

I have the same issue in terraform-provider-google.I found how to do it from Admin UI (UI -> Settings -> Features Enablement -> disable Personal compute). At the same time I didn't find how to have the same behavior from Terraform.

  • 1 kudos
Edyta
by New Contributor II
  • 4746 Views
  • 1 replies
  • 0 kudos

Resolved! Delete Databricks account

Hi everyone, as in the topic, I would like to delete unnnecesarily created account. I have found outdated solutions (e.g. https://community.databricks.com/t5/data-engineering/how-to-delete-databricks-account/m-p/6323#M2501), but they do not work anym...

  • 4746 Views
  • 1 replies
  • 0 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 0 kudos

Hi @Edyta ,FOR AWS:Manage your subscription | Databricks on AWS" Before you delete a Databricks account, you must first cancel your Databricks subscription and delete all Unity Catalog metastores in the account. After you delete all metastores associ...

  • 0 kudos
AmandaOhare
by New Contributor II
  • 1697 Views
  • 3 replies
  • 2 kudos

Resolved! Message queue to directly trigger job

Hi All,I'm very new to Databricks and trying to understand my use case a bit better.I have a databricks script / job that I want to be reactive to events outside of my databricks environment. A best case scenario would be if my script / job could aut...

  • 1697 Views
  • 3 replies
  • 2 kudos
Latest Reply
szymon_dybczak
Contributor III
  • 2 kudos

Hi @AmandaOhare ,You can use AWS Lambda to achieve that. You can setup queue trigger that will activate AWS lambda function. In that function you can call datbricks rest API that will launch workflow/job.

  • 2 kudos
2 More Replies
leo-machado
by New Contributor II
  • 582 Views
  • 1 replies
  • 3 kudos

Refresh permission on Lakeview Dashboard

Hi folks!I'm sure I'm not the only one, but our users have the tendency to click the big Refresh button on all dashboards every time they open them.Using resources efficiently is something I value deeply, so our team came up with a schedule policy - ...

image (2).png
  • 582 Views
  • 1 replies
  • 3 kudos
Latest Reply
holly
Databricks Employee
  • 3 kudos

Hi @leo-machado this is a very reasonable request. It's not feasible right now, but I'll pass it on to the product team as I'm sure you're not the only one. 

  • 3 kudos
pvilasra
by New Contributor II
  • 306 Views
  • 1 replies
  • 0 kudos

Unable to access databricls workspaces

Hi Team, Previously we are able to access databricks workspaces using username and password but now its asking for token, But we dont have access to mail account with username that we are using while login. Could you please assist on the issue how ca...

  • 306 Views
  • 1 replies
  • 0 kudos
Latest Reply
pvilasra
New Contributor II
  • 0 kudos

Anyone can help on this ?

  • 0 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels