cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

agarg
by New Contributor II
  • 3106 Views
  • 2 replies
  • 0 kudos

Databricks REST API to fetch mount points

Is there a way to fetch workspace  mount points ( mount infos) through REST API or SQL-query  ? ( similar to the python API "display(dbutils.fs.mounts())" )  I couldn't find any REST API for the mounts in the official databricks API  documentation ( ...

  • 3106 Views
  • 2 replies
  • 0 kudos
Latest Reply
pradeep1
New Contributor II
  • 0 kudos

How to create mount points using aws Databricks with shared cluster?

  • 0 kudos
1 More Replies
pv12
by New Contributor II
  • 10852 Views
  • 4 replies
  • 3 kudos

Resolved! Does REST API usage is chargeable ?

If we use REST API to perform some operation on the Databricks cluster- like running some SQL Statements, will that be chargeable, will it costs us. 

  • 10852 Views
  • 4 replies
  • 3 kudos
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 3 kudos

Yep, to sum up - anything that requires compute power and makes the cluster to spin up will be charged.

  • 3 kudos
3 More Replies
AlbertWang
by Valued Contributor
  • 5607 Views
  • 3 replies
  • 1 kudos

Resolved! Confusion about SCC, NAT, and Internet connection (Azure)

Hi all,I created an Azure Databricks workspace with the below featuresDeployed in my Azure virtual network (VNet injection)Enabled Secure cluster connectivityEnabled the Back-end Private LinkI did not create and use a NAT. I found that I still can ac...

  • 5607 Views
  • 3 replies
  • 1 kudos
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 1 kudos

Hi @AlbertWang ,If you choose SCC option, there shouldn't be NAT gateway created. You can check that yourself if you go the managed resource group that is created when you deploying databricks workspace.When you use SCC then if you would like to have...

  • 1 kudos
2 More Replies
mushroom_posse
by New Contributor II
  • 1940 Views
  • 2 replies
  • 1 kudos

Resolved! Delta sharing gold layer data within organisation but outside of Vnet

Hi all,The organisation I’m working for has a data engineering team using medallion architecture and wants to share materialised views in the gold layer to members of the organisation who do not have access to the Vnet to be delivered in power BI rep...

  • 1940 Views
  • 2 replies
  • 1 kudos
Latest Reply
mushroom_posse
New Contributor II
  • 1 kudos

Thanks for that information will take it to the infrastructure team

  • 1 kudos
1 More Replies
Mysilvercloud
by Databricks Partner
  • 1289 Views
  • 1 replies
  • 0 kudos

Databricks Account Admin architecture

Hi,I have a couple of question regarding the Databricks Account architecture. I was given an Azure admin user to create Databricks admin account. When open the Databricks admin account console, there are some metastores already & some workspaces, whi...

  • 1289 Views
  • 1 replies
  • 0 kudos
Latest Reply
Mysilvercloud
Databricks Partner
  • 0 kudos

Also another question is, I have assigned a user to be account admin, but he can not see the previews in the account console? Why is that?Ok now is work, I guess Databricks API is broken!

  • 0 kudos
know-how
by New Contributor III
  • 5435 Views
  • 6 replies
  • 4 kudos

What is the Databricks equivalent of a stored procedure that works with Power BI?

Is there a way to query Databricks using Power BI, via a stored procedure or similar? In my use case the query will have multiple parameters, that the user will provide values for at runtime. I don't think Databricks notebooks can be consumed directl...

  • 5435 Views
  • 6 replies
  • 4 kudos
Latest Reply
db_20240821
New Contributor II
  • 4 kudos

Hi @BrahmareddySorry ... But Stored procedures are much more than just views. This is a big loss for DATABRICKS users. Can they not implement something  ? SOON ? 

  • 4 kudos
5 More Replies
Rajatkumar002
by Databricks Partner
  • 3411 Views
  • 3 replies
  • 1 kudos

session expired Error

Within two minutes of logging into Databricks Community Edition, I am receiving the following error message: "Session expired." The session has ended; please log in again." I've attempted to log in again, but the message "Authenticate the session is ...

  • 3411 Views
  • 3 replies
  • 1 kudos
Latest Reply
Retired_mod
Esteemed Contributor III
  • 1 kudos

Hi @Rajatkumar002, Please review the response and let us know if it answers your question. Your feedback is valuable to us and the community. If the response resolves your issue, kindly mark it as the accepted solution. This will help close the threa...

  • 1 kudos
2 More Replies
Olfa_Kamli
by New Contributor III
  • 2219 Views
  • 1 replies
  • 0 kudos

Config sso and user provioning on databricks via terraform or api

Hi All,I am looking for a way to automate SSO configuration and user provisioning on Databricks. I see that this is feasible via the Databricks console, as documented SSO in your Databricks account console | Databricks on AWS. However, I am not sure ...

  • 2219 Views
  • 1 replies
  • 0 kudos
Latest Reply
suranamudit
New Contributor II
  • 0 kudos

Hi I am also looking to configure SSO in databricks using REST API. Were you able to figure out a way to do that?Thanks

  • 0 kudos
mickl089
by New Contributor
  • 994 Views
  • 1 replies
  • 0 kudos

Google SSO Login after renew Google SSO Cert

Hey there,SSO for Databricks was enabled and worked for us. After renewing the Google SSO Cert, it´s not possible to renew the cert also in DataBricks... can you help me?

  • 994 Views
  • 1 replies
  • 0 kudos
Latest Reply
Retired_mod
Esteemed Contributor III
  • 0 kudos

Hi @mickl089, Thank you for sharing your concern with us!   To expedite your request, please list your concerns on our ticketing portal. Our support staff would be able to act faster on the resolution (our standard resolution time is 24-48 hours).  

  • 0 kudos
clentin
by Contributor
  • 6156 Views
  • 4 replies
  • 5 kudos

Resolved! Databricks Community Edition -- Service Status Page

Databricks Community Edition Service Status Page is not available. Or if available the link is well hidden. This feature will be beneficial to all users of Databricks Community Edition. 

  • 6156 Views
  • 4 replies
  • 5 kudos
Latest Reply
Retired_mod
Esteemed Contributor III
  • 5 kudos

Hi @clentin and @szymon_dybczak, Between 22:00 UTC on August 14, 2024, and 14:43 UTC on August 19, 2024, you may have been a user of Databricks' Community Edition who experienced errors citing "Your session has expired, please authenticate again" wit...

  • 5 kudos
3 More Replies
shiva1212
by New Contributor II
  • 7172 Views
  • 3 replies
  • 1 kudos

Resolved! Shared cluster

Hello Community,At my company, we use Databricks extensively for workflow development. We operate on a shared cluster that multiple users access concurrently, and we utilize a shared workspace, where one folder is accessible to a designated group. Ea...

  • 7172 Views
  • 3 replies
  • 1 kudos
Latest Reply
Aviral-Bhardwaj
Esteemed Contributor III
  • 1 kudos

No Shiva, it don't work like that,one notebook one session simple, collaborative notebook means one development notebook with multiple user ,but multiple session wont work like that .

  • 1 kudos
2 More Replies
breaka
by New Contributor III
  • 2151 Views
  • 1 replies
  • 0 kudos

DAB Deployment to a (external) volume on UC

Hi everyone,We are currently migrating to Unity Catalog and also our CD pipeline from DBX to Assets Bundle.Reading the documentation beforehand, I was totally sure that Databricks would support deploying jobs directly to (external) volumes as it woul...

  • 2151 Views
  • 1 replies
  • 0 kudos
Latest Reply
prith
New Contributor III
  • 0 kudos

The path in the databricks.yml is with 2 slashes.. This is what I used root_path: s3://test-${var.sdlc_env}-us-east-1/dabs-artifacts/projects/.bundle/${bundle.name}/${bundle.target}Yet still I got the error Error: Path (s3:/test-dev-us-east-1/dabs-ar...

  • 0 kudos
AClarkson
by New Contributor II
  • 13583 Views
  • 3 replies
  • 0 kudos

AWS Secrets Manager access

HiWe're in the process of moving over from Databricks in Azure to AWS.I am trying to establish a method of accessing secrets from AWS Secrets Manager (we were using Azure KeyVault) and understand this can be done with boto as suggested from AWS.We ha...

  • 13583 Views
  • 3 replies
  • 0 kudos
Latest Reply
AClarkson
New Contributor II
  • 0 kudos

I should add I'm running this on a shared cluster and therefore suspect the following constraints are impeding me:Taken from:  https://docs.databricks.com/en/clusters/configure.htmlCannot connect to the instance metadata service (IMDS), other EC2 ins...

  • 0 kudos
2 More Replies
Mathias_Peters
by Contributor II
  • 1551 Views
  • 1 replies
  • 0 kudos

Databricks Terraform Module for reading the current clusters times out

Hi, we are managing our Databricks Workspaces entirely using terraform. As of today, the module to read the current clusters fails with a timeout. We use this:data "databricks_clusters" "this" { }This has worked flawlessly before. However, now we are...

Mathias_Peters_0-1723728679046.png
  • 1551 Views
  • 1 replies
  • 0 kudos
Latest Reply
Mathias_Peters
Contributor II
  • 0 kudos

Correction of my previous post: The error occurs when bumping the provider version to 1.50.0. It works fine on 1.49.0

  • 0 kudos
MichaelO
by New Contributor III
  • 1796 Views
  • 2 replies
  • 0 kudos

Terraform databricks new feature

I recently saw a new feature in databricks that allows to set an upper limit of run time for jobs, so it fails when the limit is breached. See image below:Are we able to terraform code this new feature?

MichaelO_0-1723549982471.png
  • 1796 Views
  • 2 replies
  • 0 kudos
Latest Reply
Retired_mod
Esteemed Contributor III
  • 0 kudos

Hi @Rjdudley, Thanks for reaching out! Please review the response and let us know which best addresses your question. Your feedback is valuable to us and the community.   If the response resolves your issue, kindly mark it as the accepted solution. T...

  • 0 kudos
1 More Replies