cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

clyormz
by New Contributor
  • 656 Views
  • 1 replies
  • 0 kudos

databricks on azure jdbc

Hello Databricks teamI have one question regarding data bricks on azure configuration using jdbc [Simba][SparkJDBCDriver](700100)I am getting below error message : java.sql.SQLException: [Simba][SparkJDBCDriver](700100) Connection timeout expired. De...

  • 656 Views
  • 1 replies
  • 0 kudos
Latest Reply
jose_gonzalez
Moderator
  • 0 kudos

Check your network connection. Try "%sh nc -zv {hostname} {port}"

  • 0 kudos
ossinova
by Contributor II
  • 1271 Views
  • 3 replies
  • 0 kudos

Override default Personal Compute policy using terraform / disable Personal Compute policy

I want to programmatically do some adjustments to the default personal compute resource or preferably create my own custom one based on the same configuration or policy family (in which all users can gain access to) when deploying a new workspace usi...

  • 1271 Views
  • 3 replies
  • 0 kudos
Latest Reply
feiyun0112
Contributor III
  • 0 kudos

use api to create new cluster, set autotermination_minutes parameterhttps://docs.databricks.com/api/workspace/clusters/create#autotermination_minutes 

  • 0 kudos
2 More Replies
Snoonan
by Contributor
  • 1437 Views
  • 1 replies
  • 0 kudos

Resolved! Creating Databricks workspace

Hi all,I am creating a Databricks workspace that has its own virtual network.When I create it I get this error:'The workspace 'xxxxxx' is in a failed state and cannot be launched. Please review error details in the activity log tab and retry your ope...

  • 1437 Views
  • 1 replies
  • 0 kudos
Latest Reply
Snoonan
Contributor
  • 0 kudos

Hi all,I resolved the issue.My subnets did not have the correct delegations.Thanks,Sean

  • 0 kudos
Joaquim
by New Contributor II
  • 996 Views
  • 3 replies
  • 0 kudos

New admin question: How do you enable R on a existing cluster?

Hello Community. I have a user trying to use R and receive the error message illustrated on the attachment. I can't seem to find correct documentation on enabling R on an existing cluster. Would anyone be able to point me in the right direction? Than...

  • 996 Views
  • 3 replies
  • 0 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 0 kudos

This will help you

  • 0 kudos
2 More Replies
saikrishna3390
by New Contributor II
  • 1227 Views
  • 2 replies
  • 1 kudos

Resolved! How do we get user list who accessed specific table or view in Unity catalog for last 6 months

We have a business use case where we want to track users who accessed a specific table in Unity catalog for last 6 months . Is there a way where we can pull this data ? 

  • 1227 Views
  • 2 replies
  • 1 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 1 kudos

yes system table will have all details

  • 1 kudos
1 More Replies
agarg
by New Contributor II
  • 650 Views
  • 2 replies
  • 1 kudos

Databricks REST API to fetch mount points

Is there a way to fetch workspace  mount points ( mount infos) through REST API or SQL-query  ? ( similar to the python API "display(dbutils.fs.mounts())" )  I couldn't find any REST API for the mounts in the official databricks API  documentation ( ...

  • 650 Views
  • 2 replies
  • 1 kudos
Latest Reply
agarg
New Contributor II
  • 1 kudos

Thank you for the response @Kaniz . As per your comment above I tried looking for any system catalog tables (https://docs.databricks.com/en/sql/language-manual/sql-ref-information-schema.html) that could provide the relevant information regarding the...

  • 1 kudos
1 More Replies
madhura
by New Contributor II
  • 747 Views
  • 3 replies
  • 0 kudos

Rest endpoint for data bricks audit logs

I am trying to find official documentation link to get audit logs of data bricks. unable to find it. referred on.https://docs.databricks.com/en/administration-guide/account-settings/audit-logs.htmlhttps://docs.databricks.com/api/workspace/introductio...

  • 747 Views
  • 3 replies
  • 0 kudos
Latest Reply
Yeshwanth
Honored Contributor
  • 0 kudos

@madhura  I could not find any endpoint that can be used to get the Audit logs. However, you can enable system tables in your workspace and try to read the data just like you read from any other table. Please check this to enable system tables: https...

  • 0 kudos
2 More Replies
Priyam1
by New Contributor III
  • 866 Views
  • 1 replies
  • 0 kudos

Databricks Job alerts

I'm currently running jobs on job clusters and would like these jobs to time out after 168 hours (7 days), at which point a new job cluster will be assigned. This timeout is specifically to ensure that jobs don't run on the same cluster for too long,...

  • 866 Views
  • 1 replies
  • 0 kudos
Latest Reply
Yeshwanth
Honored Contributor
  • 0 kudos

@Priyam1 Good day!Based on the information provided, it seems that we do not have a direct way to mute notifications for timed-out jobs while still receiving alerts for job failures. You can reduce the number of notifications sent by filtering out no...

  • 0 kudos
mathiaskvist
by New Contributor III
  • 3350 Views
  • 7 replies
  • 1 kudos

Resolved! Valid Workspace Conf keys

HiI'm trying to automate the configuration of Admin Settings of our Databricks Workspace using Terraform. However identifying the correct config keys is very difficult.Databricks exposes a Workspace Conf API (Enable/disable features | Workspace Conf ...

  • 3350 Views
  • 7 replies
  • 1 kudos
Latest Reply
Alexis_Chicoine
New Contributor II
  • 1 kudos

I wanted to know the key for Store interactive notebook results in customer account.It's not ideal but by using the browser dev tools you can find out what it is by looking at the network activity after toggling it in the UI.  

  • 1 kudos
6 More Replies
erigaud
by Honored Contributor
  • 7615 Views
  • 8 replies
  • 8 kudos

Resolved! Installing libraries on job clusters

Simple question : what is the way to go to install libraries on job clusters ? There does not seem to be a "Libraries" tab on the UI as opposed to regular clusters. Does it mean that the only option is to use init scripts ? 

  • 7615 Views
  • 8 replies
  • 8 kudos
Latest Reply
brushdoylt
New Contributor II
  • 8 kudos

@daniel_sahalflappy birdIs there another way to add libraries to the ADF Databricks task even faster?

  • 8 kudos
7 More Replies
RozaZaharieva
by New Contributor
  • 1067 Views
  • 2 replies
  • 1 kudos

Get Azure Databricks Account ID

Hi everyone,Is it possible with Terraform or Azure CLI or any other not manual method to get the value for Azure Databricks Account ID and not to use manual method as is described here - https://learn.microsoft.com/en-us/azure/databricks/administrati...

Administration & Architecture
azuredatabricks
iac
Terraform
  • 1067 Views
  • 2 replies
  • 1 kudos
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Hi @RozaZaharieva, To retrieve the Azure Databricks Account ID without manual intervention, you can use the following methods:   Azure CLI: You can use the Azure CLI to query the account details. Run the following command to get the account ID: a...

  • 1 kudos
1 More Replies
m997al
by Contributor
  • 1765 Views
  • 6 replies
  • 3 kudos

Azure Databricks with standard private link cluster event log error: "Metastore down"...

We have Azure Databricks with standard private link (back-end and front-end private link).We are able to successfully attach a Databricks workspace to the Databricks metastore (ADLS Gen2 storage).However, when trying to create tables in a catalog in ...

  • 1765 Views
  • 6 replies
  • 3 kudos
Latest Reply
CharlesWoo
New Contributor II
  • 3 kudos

can confirm that the approach will solve your error. Ran into a similar issue a while back.

  • 3 kudos
5 More Replies
aockenden
by New Contributor III
  • 668 Views
  • 4 replies
  • 0 kudos

Secrets ACL API Behavior Change

Hey all,Has the behavior of the Secrets ACL API changed over the last 24 hours? With no code changes on our scope-deployment pipeline, I am suddenly getting strange errors back from this endpoint.Anybody else noticing a change?Thanks,Alex

  • 668 Views
  • 4 replies
  • 0 kudos
Latest Reply
aockenden
New Contributor III
  • 0 kudos

Idk, I control the resource group myself and I don't remember ever granting or revoking contributor roles on that RG for any of these users which are now suddenly throwing errors. Interesting to see that line from the docs... I wonder if that was alw...

  • 0 kudos
3 More Replies
SunilPoluri
by New Contributor
  • 533 Views
  • 1 replies
  • 0 kudos

Create Databricks managed service principal programatically ?

For the current Databricks service principal API or the Databricks SDK, an ID is required. However, when dealing with Databricks-managed service principals, you typically only have the name. For registering with cloud providers, like Microsoft Entra ...

  • 533 Views
  • 1 replies
  • 0 kudos
Latest Reply
Yeshwanth
Honored Contributor
  • 0 kudos

@SunilPoluri, could you please provide some more context by providing screenshots?

  • 0 kudos
Fresher
by New Contributor II
  • 1217 Views
  • 4 replies
  • 1 kudos

Databricks is taking too long to run a query

I ran a simple query. It is taking too much time and it doesn't stop running. I am the only one who is using cluster. It's happening to every notebook in the workspace.

  • 1217 Views
  • 4 replies
  • 1 kudos
Latest Reply
Lakshay
Esteemed Contributor
  • 1 kudos

Do you see spark jobs running in Spark UI ?

  • 1 kudos
3 More Replies
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels