I am logged into a workspace and trying to check the schemas in legacy hive_metastore using a serverless compute, I can see the schemas listed.However, when I am creating all-purpose cluster and trying to check the schemas in legacy hive_metastore. I...
There is API available to create a secret for Service Principal./api/2.0/accounts/{account_id}/servicePrincipals/{service_principal_id}/credentials/secretsCan anyone please help what has to be passed as authentication for this API ? This is looking a...
hi @AnkurMittal008 In general, I do not recommend using tokens anymore. Instead, if you want to log in via databricks cli, you can use this command:databricks auth login --host https://accounts.azuredatabricks.net/ --account-id <YOUR_ACCOUNT_ID>This ...
Scenario - I have an ADLS g2 account and trying to read a delta table using ABFS driver. I am using Databricks Serverless compute. There are no firewalls in place as I am working with sample data. There's network line of sight between Databricks serv...
Hi @jeet414,You can always read Delta table from ADLS storage in Databricks mount points: https://learn.microsoft.com/en-us/azure/databricks/dbfs/mountsThis is just experimental, please give it a try.Best Regards,Nivethan V
What Pods and Services subranges would you recommend for a /22 subnet for a custom VPC for a new GCP Databricks deployment in the GCE era?
When trying to query a view, my_view, we sometimes see a spurious error. This seems to occur after the table underlying the view has been updated. The error persists for a while and then it seems to fix itself. Error running query: [42501] [Simba][Ha...
Are there DDL updates of the view going on?
Hi Team,The GCP Databricks URL https://accounts.gcp.databricks.com/ for GCP Databricks is linked to the GCP Billing Account.We have two clients with separate GCP Organizations:client1.example.comclient2.example.comBoth GCP Organizations share the sam...
@karthiknuvepro The Databricks Account should be handled by a third-party Cloud Administration team. The workspace admins can work with them to set up the necessary cloud resources to support their catalogs and user adds/remove from their selected a...
Say we have completed the migration of tables from Hive Metastore to UC. All the users, jobs and clusters are switched to UC. There is no more activity on Legacy Hive Metastore.What is the best recommendation on deleting or cleaning the Hive Metastor...
Hi @bhanu_dp,If I understood correctly, all the tables / objects in your HIVE METASTORE is now migrated to UC Catalog. If you have used UCX for migration, you might not see the process / workflow for removing the HIVE METASTORE from your workspace as...
Privileged Identity Management (PIM) can be used to secure access to critical Databricks roles with Just-in-Time (JIT) access. This approach helps organizations enforce time-bound permissions, approval workflows, and centralized auditing for sensitiv...
Thanks for sharing this, it is helpful. However, onboarding the AAD group as an account admin under the Databricks account is not straightforward and is also not clearly explained in the blog.
I have some questions regarding Databricks App.1) Can we use Framework other than mentioned in documentation( Streamlit,Flask,Dash,Gradio,Shiny).2) Can we allocate compute more than 2 vCPU and 6GB memory to any App.3) Any other programming language o...
1.) You can use most Python-based application frameworks, including some beyond those mentioned above.(Reference here) 2.) Currently, app capacity is limited to 2 vCPUs and 6 GB of RAM. However, future updates may introduce options for scaling out an...
Databricks Apps allows us to define dependencies & an entrypoint to execute a Python application like Gradio, Streamlit, etc. It seems I can also run a FastAPI application and access via an authenticated browser which is potentially a very powerful c...
Hello, Thank you for your questions and answers regarding this topic. Is it available this feature right now? Or still not supported? Thank you in advance
Hi Databricks community. I have 2 questions that I'd appreciate if you can shed some lights on:Is the new Network Policies in Databricks account, only applicable to serverless compute or are these workspace-wide policies which apply to all other comp...
I am not support, just a regular customer like you, but here is what I know:1. Yes, serverless egress only applies to serverless. There is another upcoming change you'll need to make for your classic compute, announced by Microsoft at Default outbou...
Hi,I have taken a Lab Subscription from databricks academy. It worked fine for me for 1 day. As I try to login today, it is not starting up. I really appreciate any help in this regard. I have things as planned to do on the lab and when it is not sta...
Hi, was this resolved? can I know the solution please? Also, is there any restriction on the number of days or so to use this lab? I am getting 'Session ended' message even after I login again.RegardsDeepa
Hi everyone,I'm new to Databricks and am trying to connect my Google Cloud Storage bucket to my Databricks workspace. I have a 43GB CSV file stored in a GCP bucket that I want to work with. Here’s what I've done so far:Bucket Setup:I created a GCP bu...
I am in the process of setting up a new Databricks account for AWS commercial. I mistakenly setup the account with the email: databricks-external-nonprod-account-owner@slingshotaerospace.com to not be self-managed and I would like for this new accoun...
Or better yet if we could delete it so I can re-create the account.
HiI am trying to establish a method of accessing secrets from AWS Secrets Manager and understand this can be done with boto as suggested from AWS.We have created all of the relevant IAM roles, instance profiles etc. Accessing S3 with this method is ...
Hi @Alberto_Umana,Yes, the Role has the SecretsManagerReadWrite policy.Also within my furthet investigation I tried running it via Personal Cluster and it worked!Basically, 3 scenarios:- Shared Cluster with applied InstanceProfile - Secrets failing- ...
