cancel
Showing results for 
Search instead for 
Did you mean: 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

KBN
by New Contributor II
  • 13569 Views
  • 2 replies
  • 2 kudos

Authentication for Databricks Apps

Databricks Apps allows us to define dependencies & an entrypoint to execute a Python application like Gradio, Streamlit, etc. It seems I can also run a FastAPI application and access via an authenticated browser which is potentially a very powerful c...

  • 13569 Views
  • 2 replies
  • 2 kudos
Latest Reply
apinara
New Contributor II
  • 2 kudos

Hello, Thank you for your questions and answers regarding this topic. Is it available this feature right now? Or still not supported? Thank you in advance

  • 2 kudos
1 More Replies
mohsen-dbx
by New Contributor II
  • 3905 Views
  • 2 replies
  • 1 kudos

Databricks Network Policies

Hi Databricks community. I have 2 questions that I'd appreciate if you can shed some lights on:Is the new Network Policies in Databricks account, only applicable to serverless compute or are these workspace-wide policies which apply to all other comp...

mohsendbx_0-1740670036975.png
  • 3905 Views
  • 2 replies
  • 1 kudos
Latest Reply
Rjdudley
Honored Contributor
  • 1 kudos

I am not support, just a regular customer like you, but here is what I know:1. Yes, serverless egress only applies to serverless.  There is another upcoming change you'll need to make for your classic compute, announced by Microsoft at Default outbou...

  • 1 kudos
1 More Replies
dawnCoder
by New Contributor III
  • 2472 Views
  • 3 replies
  • 1 kudos

Vocareum Lab issue

Hi,I have taken a Lab Subscription from databricks academy. It worked fine for me for 1 day. As I try to login today, it is not starting up. I really appreciate any help in this regard. I have things as planned to do on the lab and when it is not sta...

  • 2472 Views
  • 3 replies
  • 1 kudos
Latest Reply
Deepa710
New Contributor III
  • 1 kudos

Hi, was this resolved? can I know the solution please? Also, is there any restriction on the number of days or so to use this lab? I am getting 'Session ended' message even after I login again.RegardsDeepa

  • 1 kudos
2 More Replies
patrykp
by New Contributor II
  • 853 Views
  • 2 replies
  • 0 kudos

AWS Secrets Manager access

HiI am trying to establish a method of accessing secrets from AWS Secrets Manager and understand this can be done with boto as suggested from AWS.We have created all of the relevant IAM roles, instance profiles etc.  Accessing S3 with this method is ...

  • 853 Views
  • 2 replies
  • 0 kudos
Latest Reply
patrykp
New Contributor II
  • 0 kudos

Hi @Alberto_Umana,Yes, the Role has the SecretsManagerReadWrite policy.Also within my furthet investigation I tried running it via Personal Cluster and it worked!Basically, 3 scenarios:- Shared Cluster with applied InstanceProfile - Secrets failing- ...

  • 0 kudos
1 More Replies
Dnirmania
by Contributor
  • 891 Views
  • 1 replies
  • 0 kudos

Communication between multiple region workspaces & metastore

Hi EveryoneWe currently have a workspace and metastore in the EastUS region, and we’re planning to set up another workspace and metastore in the Canada region. Additionally, we need to be able to access data from the Canada region within the US regio...

  • 891 Views
  • 1 replies
  • 0 kudos
Latest Reply
Advika_
Databricks Employee
  • 0 kudos

Hello @Dnirmania! Delta Sharing is ideal for read-only, cross-platform data access without duplication. Direct metastore connections offer low-latency access between Databricks workspaces under a unified governance model. Additionally, you may explor...

  • 0 kudos
ScottDunk
by New Contributor
  • 814 Views
  • 1 replies
  • 0 kudos

Query not returning result - no internet connection

Hi,New to DBricks. We have set up our new environment and imported some data to tables.  However when we run a query against the table using a newly created warehouse we can see the query runs fine but throw the error: failed communicating with serve...

  • 814 Views
  • 1 replies
  • 0 kudos
Latest Reply
emillion25
New Contributor III
  • 0 kudos

It sounds like there might be a network configuration issue or a client-side problem preventing your query results from appearing in the session, even though they're executing successfully on the server. Here are a few things you could check:Network ...

  • 0 kudos
neointab
by New Contributor
  • 2551 Views
  • 2 replies
  • 2 kudos

Restrict a Workspace User from Creating/Managing Databricks Jobs

Hello Databricks team,I currently have a workspace user, and I want to disable their ability to create or manage Databricks jobs entirely. Specifically, I would like to prevent the user from accessing the "Create Job" option in the Databricks UI or v...

  • 2551 Views
  • 2 replies
  • 2 kudos
Latest Reply
JameDavi_51481
Contributor
  • 2 kudos

Are there any updates on any of these internal feature requests? It's a pretty big failure in the permissions model that we can't prevent users from scheduling arbitrary workloads on compute they are able to access.

  • 2 kudos
1 More Replies
saiV06
by New Contributor III
  • 3294 Views
  • 0 replies
  • 0 kudos

Lakehouse Federation - Unable to connect to Snowflake using "PEM Private Key"

Hi,I'm currently using Lakehouse Federation feature on databricks to run queries against Snowflake datawarehouse. Today I'm using a service credential to establish the connection (user id & pwd), but I have to change it to use private key. I tried us...

Administration & Architecture
FederatedQuery
LakehouseFederation
SnowflakeConnection
  • 3294 Views
  • 0 replies
  • 0 kudos
spd_dat
by New Contributor III
  • 916 Views
  • 2 replies
  • 0 kudos

Can AWS workspaces share subnets?

The docs state:"You can choose to share one subnet across multiple workspaces or both subnets across workspaces."as well as:"You can reuse existing security groups rather than create new ones."and on this page:"If you plan to share a VPC and subnets ...

  • 916 Views
  • 2 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @spd_dat, I will check with the team internally and try to replicate the same in my workspace. This should be possible as documentation indicates. Based on the error you are hitting, could you please share the configuration you are setting in your...

  • 0 kudos
1 More Replies
Ashok_AWS
by New Contributor
  • 504 Views
  • 1 replies
  • 0 kudos

Databricks SSO enableed to Azure AD and this set up was deleted

HI Team,The SSO was enabled on the Databricks account with Azure AD and the environment is on AWS platform.The enterprise application which was used is been deleted from azure AD. No emergency user access has been set up.It is locked up, possible to ...

  • 504 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @Ashok_AWS, Could you file a case with us, we can help you. Send a mail to: help@databricks.comhttps://docs.databricks.com/aws/en/resources/support

  • 0 kudos
Inna_M
by New Contributor III
  • 1549 Views
  • 3 replies
  • 2 kudos

Question about moving to Serverless compute

Hi my organization is using Databricks in Canada Est region and Serverless isn't available in our region yet? at all?I would like to know if it is worth the effort to change region for the Canada Central, where Serverless compute is available. We do ...

  • 1549 Views
  • 3 replies
  • 2 kudos
Latest Reply
Inna_M
New Contributor III
  • 2 kudos

hi Takuya Omi, thank you for responding. My question is: if we are to migrate our existing workspaces (3) and UC to Canada Central. Is it doable? Is it worth it? What does it imply? What are the best practices to do so?Thank you. 

  • 2 kudos
2 More Replies
Bayees
by New Contributor II
  • 945 Views
  • 2 replies
  • 0 kudos

Removing storage account location from metastore fails

I am trying to remove the storage account location for our UC metastore. I am getting the error:I have tried assigning my user and service principal permission to create external location.

Bayees_0-1739286165573.png
  • 945 Views
  • 2 replies
  • 0 kudos
Latest Reply
Bayees
New Contributor II
  • 0 kudos

I accomplished it by creating a storage account credential and external location manually. Then I was able to able the remove the metastore path.What then happen was that the path for __databricks_internal catalog was set to the storage account that ...

  • 0 kudos
1 More Replies
spd_dat
by New Contributor III
  • 2362 Views
  • 2 replies
  • 0 kudos

Resolved! How does a non-admin user read a public s3 bucket on serverless?

As an admin, I can easily read a public s3 bucket from serverless:spark.read.parquet("s3://[public bucket]/[path]").display()So can a non-admin user, from classic compute. But why does a non-admin user, from serverless (both environments 1 & 2) get t...

  • 2362 Views
  • 2 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @spd_dat, Is the S3 bucket in the same region as your workspace? It might required using a IAM role / S3 bucket to allow the bucket even if it is public. Just for a test can you try giving the user who is trying the below permission: GRANT SELECT ...

  • 0 kudos
1 More Replies
Wayne
by New Contributor III
  • 730 Views
  • 1 replies
  • 0 kudos

AWS custom role for Databricks clusters - no instance profile ARN

Try to follow the instructions to create custom IAM role for EC2 instance in Databricks clusters, but I can't find the instance profile ARN on the role. If I create a regual IAM role on EC2, I can find both role ARN and instance profile ARN.https://d...

  • 730 Views
  • 1 replies
  • 0 kudos
Latest Reply
Takuya-Omi
Valued Contributor III
  • 0 kudos

@Wayne I need to understand more about what you’re trying to achieve,but if you’re looking to grant permissions to the EC2 instances running behind a Databricks cluster using an instance profile, the following documentation provides a detailed explan...

  • 0 kudos
Anonymous
by Not applicable
  • 11817 Views
  • 2 replies
  • 0 kudos
  • 11817 Views
  • 2 replies
  • 0 kudos
Latest Reply
Ryan_Chynoweth
Databricks Employee
  • 0 kudos

You can get the job details from the jobs get api, which takes the job id as a parameter. This will give you all the information available about the job, specifically the job name. Please note that there is not a field called "job description" in the...

  • 0 kudos
1 More Replies