cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

timlopez
by New Contributor
  • 1480 Views
  • 1 replies
  • 0 kudos

AWS S3 Object Lock Timeline

Hello, I see that Object Lock is not currently supported by Databricks: https://kb.databricks.com/en_US/delta/object-lock-error-write-delta-s3Is there any timeline / roadmap for the support of this feature?

  • 1480 Views
  • 1 replies
  • 0 kudos
Latest Reply
arpit
Databricks Employee
  • 0 kudos

@timlopez We already have a feature request created on this one. Our product team is currently reviewing this feature request and will decide further on this feature in the product roadmap.

  • 0 kudos
User16752244127
by Contributor
  • 1589 Views
  • 1 replies
  • 0 kudos
  • 1589 Views
  • 1 replies
  • 0 kudos
Latest Reply
arpit
Databricks Employee
  • 0 kudos

@User16752244127 Please elaborate on what do you mean by "features". I am tagging a generic documentation, but if you are looking for something specific please let me know.

  • 0 kudos
_rafa
by New Contributor
  • 1921 Views
  • 1 replies
  • 0 kudos

Azure SP OAuth wont generate token

Hi all,Hopefully someone has had this issue and fixed it before. No matter how many times I try, no token is generated (yes, i am in a premium tier). Any ideas? I tried using the API but I got this error that apparently is because I am in Azure. for ...

_rafa_0-1702294084413.png _rafa_1-1702294208009.png _rafa_0-1702294866329.png _rafa_1-1702294945196.png
  • 1921 Views
  • 1 replies
  • 0 kudos
Latest Reply
Yeshwanth
Databricks Employee
  • 0 kudos

Hello @_rafa  ,Currently, we do not have an API similar to the "https://<databricks-instance>/api/2.0/token-management/on-behalf-of/tokens" in azure as Azure supports the service principle authentication by default. Please authenticate to normal PAT ...

  • 0 kudos
Wojciech_BUK
by Valued Contributor III
  • 3674 Views
  • 1 replies
  • 0 kudos

Azure Databricks SQL Warehouse tags not being populated

In Azure cost monitor I am trying to get list of my cluster names and cost associated to it.It works fine with job cluster and all purpos clusters but not for SQL Warehouses.When you look at below example, where I am querying only for my SQL Warehous...

Wojciech_BUK_0-1703773554117.png Wojciech_BUK_1-1703773672483.png
Administration & Architecture
azure
cost
monitoring
SQL warehouse
tags
  • 3674 Views
  • 1 replies
  • 0 kudos
Latest Reply
Yeshwanth
Databricks Employee
  • 0 kudos

@Wojciech_BUK I would recommend you get in touch with the Azure Pricing team once to check this in a great deatil.

  • 0 kudos
dbx_8451
by New Contributor II
  • 1516 Views
  • 1 replies
  • 0 kudos

How to delete or update cluster-scoped init scripts

We are working on to deprecate the dbfs based cluster-scoped init scripts and replace them with workspace based, therefore finding a way to delete the dbfs cluster-scoped init scripts from all the clusters that are running in our environment using RE...

  • 1516 Views
  • 1 replies
  • 0 kudos
Latest Reply
arpit
Databricks Employee
  • 0 kudos

@dbx_8451 There is no direct way for doing it, you may need to automate using the list Cluster API.

  • 0 kudos
bpark
by New Contributor
  • 2359 Views
  • 1 replies
  • 0 kudos

No enable option for Customize containers with Databricks Container Services on Azure and GCP

Customize containers with Databricks Container ServicesI want to create a spark cluster with the hail container [1]. But there is no Enable Container Services on the Workspace configuration for Azure Databricks and GCP Databricks.The API does not ret...

  • 2359 Views
  • 1 replies
  • 0 kudos
Latest Reply
arpit
Databricks Employee
  • 0 kudos

Did you tested out using the cluster API ?

  • 0 kudos
alesventus
by Contributor
  • 2045 Views
  • 1 replies
  • 0 kudos

Change users status in account console

One of our users has status = Inactive in Admin setting. I tried to set it to active using rest api and the documentation but seems like api is not changing status.I tested api and I can list users so api setting are ok.api request: import requests i...

  • 2045 Views
  • 1 replies
  • 0 kudos
Latest Reply
arpit
Databricks Employee
  • 0 kudos

@alesventus Can you test the same using CLI or may be try to make the status as inactive for a dummy user? Just want to validate if the API is actually working for you or not.

  • 0 kudos
ashdam
by New Contributor III
  • 1641 Views
  • 1 replies
  • 0 kudos

Installing databricks CLI on windows . sh ??¿?

I am following databricks documentaiton for installing the cli on windows and i found that uses sh command which of course doesnt work.My windows in admin mode says cannot find "sh" application. I could install cgwyn and tools like that but since the...

ashdam_0-1698741976775.png ashdam_1-1698742182892.png
  • 1641 Views
  • 1 replies
  • 0 kudos
Latest Reply
arpit
Databricks Employee
  • 0 kudos

Here is the doc which you can refer:AWS, GCP, and Azure 

  • 0 kudos
Kenthos
by New Contributor II
  • 3893 Views
  • 1 replies
  • 1 kudos

Resolved! Guide to Migrating from AWS to Azure

Hello, I would like to know if we have any documents, checklists, or step-by-step instructions for migrating from Databricks on AWS to Azure.

  • 3893 Views
  • 1 replies
  • 1 kudos
Latest Reply
arpit
Databricks Employee
  • 1 kudos

Yes, Databricks Migration Tool is available on Github

  • 1 kudos
cesarv
by New Contributor
  • 1247 Views
  • 1 replies
  • 0 kudos

Error upgrading Databricks workspace to premium

I tried to update a databricks workspace from Standard to Premium in the Azure portal clicking "Upgrade to Premium" but even when it was supposedly "upgraded", I still can't restrict the use of clusters or modify permissions (I'm an administrator on ...

  • 1247 Views
  • 1 replies
  • 0 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 0 kudos

You might need to open a support ticket with Azure to get more details on this issue

  • 0 kudos
anurag_dev
by New Contributor
  • 2250 Views
  • 1 replies
  • 0 kudos

getting alerts in AWS UnauthorizedAccess:IAMUser/InstanceCredentialExfiltration.InsideAWS

Hi folks, I have been using IAM instance profile as a credentials provider to Databricks workspace. I am getting this error "Credentials created exclusively for an EC2 instance using instance role <role-arn> have been used from a remote AWS account 4...

  • 2250 Views
  • 1 replies
  • 0 kudos
Latest Reply
Yeshwanth
Databricks Employee
  • 0 kudos

Hi @anurag_dev  Databricks runs a commit service that coordinates writes to Amazon S3 from multiple clusters. This service runs in the Databricks control plane. (441371677306)The S3 commit service helps guarantee consistency of writes across multiple...

  • 0 kudos
bas
by New Contributor III
  • 10147 Views
  • 9 replies
  • 2 kudos

Resolved! Adding a service principal to workspace fails

Trying to add an Azure Service Principal to a workspace fails, whether via the web UI or the databricks cli. With the databricks, the console gives this:databricks service-principals -p ************* create --application-id ********-****-****-****-**...

  • 10147 Views
  • 9 replies
  • 2 kudos
Latest Reply
brickmaker
New Contributor II
  • 2 kudos

Same issue in AWS Databricks.

  • 2 kudos
8 More Replies
sl79
by New Contributor II
  • 2673 Views
  • 1 replies
  • 1 kudos

Unable to start a SQL cluster

We have a GCP databricks hosted deployment. last night our SQL warehouse failed to start and now just sits with this message:"Cluster Start-up Delayed. Please wait while we continue to try and start the cluster. No action is required from you"It didn...

  • 2673 Views
  • 1 replies
  • 1 kudos
Latest Reply
jose_gonzalez
Databricks Employee
  • 1 kudos

Could you share the driver logs if you have them??

  • 1 kudos