cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Lakehouse Architecture


Forum Posts

rfreitas
by New Contributor II
  • 4775 Views
  • 4 replies
  • 0 kudos

Resolved! SCIM Synchronization for Email Change Cases in Azure AD

Hi everyone,I would like to know if the following behavior is expected or if it is a misconfiguration in SCIMWe are going through a change in the email of some users. So we did a test, changing the email of one of them, but the result was not OK beca...

  • 4775 Views
  • 4 replies
  • 0 kudos
Latest Reply
Ayushi_Suthar
Databricks Employee
  • 0 kudos

Hi @rfreitas , It is an expected behaviour because we don't support email or username updates/edits in the SCIM as of now. As per Databricks documentation, an email update is not supported in SCIM. You cannot update the username or email address of a...

  • 0 kudos
3 More Replies
danmlopsmaz
by New Contributor II
  • 12450 Views
  • 4 replies
  • 0 kudos

Resolved! Databricks job creator update with API

Hi team,Greetings.Do you know if there is a way to update the creator of a databricks_job using the API? the Documentation does not show "the creator" property and when I tried setting the creator, this property is not updated in the workspace UI.The...

  • 12450 Views
  • 4 replies
  • 0 kudos
Latest Reply
Ayushi_Suthar
Databricks Employee
  • 0 kudos

Hi @danmlopsmaz , Thanks for bringing up your concerns, always happy to help  I understand the customer wanted to change the creator of the job but at this moment we could change the owner but not the creator. But you should be able to clone an exist...

  • 0 kudos
3 More Replies
edmundsecho
by New Contributor II
  • 12986 Views
  • 2 replies
  • 0 kudos

how register a published app

Is there a way to register a web app with Databricks to enable it to access a user's data using OAuth2? i.e., be included in the list of published apps with a pre-negotiated scope? 

  • 12986 Views
  • 2 replies
  • 0 kudos
Latest Reply
Yeshwanth
Databricks Employee
  • 0 kudos

@edmundsecho could you please elaborate more on the use case? 

  • 0 kudos
1 More Replies
Carsten03
by New Contributor III
  • 4064 Views
  • 7 replies
  • 2 kudos

Resolved! CloudWatch Agent Init Script Fails

Hi,I am trying to install the CloudWatch log agent on my cluster, using this tutorial from AWS https://aws.amazon.com/blogs/mt/how-to-monitor-databricks-with-amazon-cloudwatch/They provide an init script there but when I try to start my cluster I get...

  • 4064 Views
  • 7 replies
  • 2 kudos
Latest Reply
alexbishop
New Contributor II
  • 2 kudos

Hi @Yeshwanth I have the exact same issue here. I have tried to update my cluster with the init script that you kindly shared in this thread. However, my error is still:Init script failure:Cluster scoped init script s3://databricks-init-scripts-caden...

  • 2 kudos
6 More Replies
VJ3
by New Contributor III
  • 7612 Views
  • 0 replies
  • 0 kudos

workspace local group once workspace has been migrated/Associated to Unity Catalog

Hello,Once we associate/migrate Databricks workspace to Unity Catalog, all the workspaces group will sync with Unity Catalog and workspace group would be renamed to workspace-local group. Databricks recommends to remove workspace-local group. Do we k...

  • 7612 Views
  • 0 replies
  • 0 kudos
Datazilla
by New Contributor III
  • 8692 Views
  • 5 replies
  • 1 kudos

Is a central UC Catalog management a Good Practice?

I am working at large company with many more or less independent divisions and we are currently working on the roll out of Unity Catalog in Azure. The idea was to have a central infrastructure repository (deployed via Terraform) to manage all central...

  • 8692 Views
  • 5 replies
  • 1 kudos
Latest Reply
Datazilla
New Contributor III
  • 1 kudos

Workspace Admins: Consider configuring permissions for workspace admins in the Account console to strike a balance between autonomy and governance.@Retired_mod Do you have any information about this configuration? I cannot find such thing in the Acco...

  • 1 kudos
4 More Replies
sheridan06
by New Contributor III
  • 1202 Views
  • 2 replies
  • 0 kudos

Resolved! Architecture With the Enhanced Security and Compliance Add-On

Morning everyone!  Wondering if someone has a high-level reference architecture of Databricks on AWS that includes the Enhanced Security and Compliance Add-On??thanks,Brad

  • 1202 Views
  • 2 replies
  • 0 kudos
Latest Reply
Yeshwanth
Databricks Employee
  • 0 kudos

@sheridan06 please check this documentation: https://www.databricks.com/trust/security-features/protect-your-data-with-enhanced-security-and-compliance Let us know if it helps.

  • 0 kudos
1 More Replies
prashantjjain33
by New Contributor II
  • 1427 Views
  • 2 replies
  • 0 kudos

Disabled S3 bucket version, Next steps?

We have been using Databricks for some time and didn't knew that S3 bucket versioning was not recommended. We have disabled it now. What are the next steps that we need to take to clean up the data, should we create a lifecycle rule to delete older v...

  • 1427 Views
  • 2 replies
  • 0 kudos
Latest Reply
prashantjjain33
New Contributor II
  • 0 kudos

Adding lifecycle policies to databricks s3 bucket does not impact existing data, is that correct @Yeshwanth .

  • 0 kudos
1 More Replies
Krubug
by New Contributor
  • 800 Views
  • 1 replies
  • 0 kudos

Resolved! Set Deafult Cluster

HelloIs there a way to set one of the clusters in my workspace to be the default cluster when choosing from list of available clusters when running my Code?Thanks in Advanced.

  • 800 Views
  • 1 replies
  • 0 kudos
Latest Reply
Ayushi_Suthar
Databricks Employee
  • 0 kudos

Hi @Krubug ,Thanks for bringing up your concerns, always happy to help  Upon checking internally, it seems that we do not currently support setting a default cluster for the workspace. But we do have a Feature request titled "Workspace - Settings - S...

  • 0 kudos
Matt4
by New Contributor III
  • 2113 Views
  • 4 replies
  • 1 kudos

Resolved! Azure Databricks Assistant and Private Link

Hi,I was wondering if anyone is using the Databricks Assistant using the standard private link configuration of their network.Kind of curious how data is passed from the Databricks Workspace, in a private link configuration, to the Azure OpenAI servi...

Administration & Architecture
assistant
privatelink
  • 2113 Views
  • 4 replies
  • 1 kudos
Latest Reply
Matt4
New Contributor III
  • 1 kudos

I asked our SA at Databricks. AWS is different if anyone is looking for that answerAzure Open AI connects to the control planeIt doe not use private endpoints, though since you are an Azure customer, the traffic stays on the Azure global network (no ...

  • 1 kudos
3 More Replies
Erik
by Valued Contributor II
  • 1875 Views
  • 1 replies
  • 1 kudos

Feature request: Run cluster in a different region

We are one of those which unfortunately decided (earlier) to set up our infastructure in one of the sub-par Azure regions, and we happen to dont have any GPU clusters available.We would like to test out some GPU enhanced ML in databricks, but at the ...

  • 1875 Views
  • 1 replies
  • 1 kudos
Latest Reply
Yeshwanth
Databricks Employee
  • 1 kudos

@Erik  I think it would be really useful if we could create a cluster in another region from our normal workspace This does not seem like a possible thing for me as the Control plane resources and the Data plane resources are bound to that region. If...

  • 1 kudos
stefankoch-OLD
by New Contributor II
  • 1884 Views
  • 4 replies
  • 0 kudos

Language Preferences changes all day

Since a few days I have the phenomenon that the language in the Databricks Workspace always changes to German. I have to change the language back to English in the settings every day. My PC has German set as the default language, but I want to keep t...

  • 1884 Views
  • 4 replies
  • 0 kudos
Latest Reply
LaurentLeturgez
Databricks Employee
  • 0 kudos

Hi @stefankoch-OLD  I had the same issue from a french perspective   To fix this issue, I had to configure the Language + Region settings on the Azure portal, and set the Regional format to English (Europe) ... it was previously on France.  After a ...

  • 0 kudos
3 More Replies
orlik
by New Contributor II
  • 7155 Views
  • 3 replies
  • 4 kudos

Databricks Runtime JDK 17 upgrade

HeyI'm using Databricks Runtime 13.2 https://docs.databricks.com/en/release-notes/runtime/13.2.html#system-environmentIt uses JDK 8. Question: Is it possible to upgrade the Java version to JDK 17?Thanks!

  • 7155 Views
  • 3 replies
  • 4 kudos
Latest Reply
Venugopal
New Contributor II
  • 4 kudos

@orlik  @sroeena34 JDK 17 is in public preview for Databricks runtime versions 13.1 and above. Pls refer to this https://docs.databricks.com/en/dev-tools/sdk-java.html#create-a-cluster-that-uses-jdk-17

  • 4 kudos
2 More Replies
GS24
by New Contributor II
  • 1758 Views
  • 2 replies
  • 0 kudos

Connecting to Azure Databricks deployed using Vnet injection over Public Internet

I'm trying to connect to Azure Databricks (deployed using Vnet injection method) from a 3rd party service running on Azure in the same region. When I try to connect using the Databricks hostname directly in my connection the host name always resolves...

Administration & Architecture
Azure Private Link
Databricks
Vnet Injection
  • 1758 Views
  • 2 replies
  • 0 kudos
Latest Reply
GS24
New Contributor II
  • 0 kudos

@arpit thanks for the link and I did review the doc. But it doesn't really say whether it its always required to use an Azure Private Link to connect to databricks deployed with Vnet injection method from a 3rd party service also on deployed on Azure...

  • 0 kudos
1 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels