- 1595 Views
- 2 replies
- 0 kudos
Mismatch of Columns in databricks vs Athena.
We are trying to expose one of our external tables to databricks via unity catalog, but we are having an issue with column mismatch, ie few of our columns are not visible in databricks. Is this a known issue? If so, can anyone best advise me on where...
- 1595 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @DavidSzedlak ,1. Unity Catalog caches metadata for performance. If new columns were added to the source table after the initial creation, they may not be reflected.Run the following command to refresh metadata: ALTER TABLE <catalog>.<schema>.<ta...
- 0 kudos
- 1172 Views
- 3 replies
- 0 kudos
Timeout on docker pull in Databricks Container Services
Hello,There is a timeout that limits the size of images used in Docker Container Service. When using images containing large ML libraries, the size often exceeds the limit that could be pulled. Is there any plan to add parametrization of this timeout...
- 1172 Views
- 3 replies
- 0 kudos
- 3330 Views
- 8 replies
- 0 kudos
How do I simply disable someone's user account
I'm trying to do something seemingly very simple - disable someone's user account. I don't even want to delete the user, just disable it for the time being. How do I go about doing that?
- 3330 Views
- 8 replies
- 0 kudos
- 0 kudos
In Databricks, go to the Admin Console.Navigate to the Service Principals section.Create a Service Role (make sure it has necessary permissions (Admin Access).Generate OAuth TokenFollow the instructions in the Databricks documentation to generate ap...
- 0 kudos
- 2482 Views
- 2 replies
- 0 kudos
Proper way to collect Statement ID from JDBC Connection
Hi, We are executing DML calls on Databricks SQL Warehouse programmatically, with Java and Python.There can be thousands of executions running on daily level, so in case of an error occurs, it would be very beneficial to spot the Statement ID of the ...
- 2482 Views
- 2 replies
- 0 kudos
- 0 kudos
Found a way to extract it for below dbx-java library.-> databricks-jdbc library version : `2.6.32` private static String extractQueryIdFromDbxStatement(Hive42PreparedStatement statement) { byte[] guid = ((HiveJDBCNativeQueryExecutor) state...
- 0 kudos
- 1250 Views
- 1 replies
- 0 kudos
Secret Creation for Service Principal using API
There is API available to create a secret for Service Principal./api/2.0/accounts/{account_id}/servicePrincipals/{service_principal_id}/credentials/secretsCan anyone please help what has to be passed as authentication for this API ? This is looking a...
- 1250 Views
- 1 replies
- 0 kudos
- 0 kudos
hi @AnkurMittal008 In general, I do not recommend using tokens anymore. Instead, if you want to log in via databricks cli, you can use this command:databricks auth login --host https://accounts.azuredatabricks.net/ --account-id <YOUR_ACCOUNT_ID>This ...
- 0 kudos
- 765 Views
- 1 replies
- 0 kudos
Reading delta table from ADLS gen2 using ABFS driver
Scenario - I have an ADLS g2 account and trying to read a delta table using ABFS driver. I am using Databricks Serverless compute. There are no firewalls in place as I am working with sample data. There's network line of sight between Databricks serv...
- 765 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @jeet414,You can always read Delta table from ADLS storage in Databricks mount points: https://learn.microsoft.com/en-us/azure/databricks/dbfs/mountsThis is just experimental, please give it a try.Best Regards,Nivethan V
- 0 kudos
- 519 Views
- 1 replies
- 0 kudos
Querying view gives spurious error
When trying to query a view, my_view, we sometimes see a spurious error. This seems to occur after the table underlying the view has been updated. The error persists for a while and then it seems to fix itself. Error running query: [42501] [Simba][Ha...
- 519 Views
- 1 replies
- 0 kudos
- 0 kudos
Are there DDL updates of the view going on?
- 0 kudos
- 1248 Views
- 3 replies
- 0 kudos
Databricks Workspace Access and Permissions
Hi Team,The GCP Databricks URL https://accounts.gcp.databricks.com/ for GCP Databricks is linked to the GCP Billing Account.We have two clients with separate GCP Organizations:client1.example.comclient2.example.comBoth GCP Organizations share the sam...
- 1248 Views
- 3 replies
- 0 kudos
- 0 kudos
@karthiknuvepro The Databricks Account should be handled by a third-party Cloud Administration team. The workspace admins can work with them to set up the necessary cloud resources to support their catalogs and user adds/remove from their selected a...
- 0 kudos
- 13466 Views
- 2 replies
- 2 kudos
Authentication for Databricks Apps
Databricks Apps allows us to define dependencies & an entrypoint to execute a Python application like Gradio, Streamlit, etc. It seems I can also run a FastAPI application and access via an authenticated browser which is potentially a very powerful c...
- 13466 Views
- 2 replies
- 2 kudos
- 2 kudos
Hello, Thank you for your questions and answers regarding this topic. Is it available this feature right now? Or still not supported? Thank you in advance
- 2 kudos
- 3795 Views
- 2 replies
- 1 kudos
Databricks Network Policies
Hi Databricks community. I have 2 questions that I'd appreciate if you can shed some lights on:Is the new Network Policies in Databricks account, only applicable to serverless compute or are these workspace-wide policies which apply to all other comp...
- 3795 Views
- 2 replies
- 1 kudos
- 1 kudos
I am not support, just a regular customer like you, but here is what I know:1. Yes, serverless egress only applies to serverless. There is another upcoming change you'll need to make for your classic compute, announced by Microsoft at Default outbou...
- 1 kudos
- 2370 Views
- 3 replies
- 1 kudos
Vocareum Lab issue
Hi,I have taken a Lab Subscription from databricks academy. It worked fine for me for 1 day. As I try to login today, it is not starting up. I really appreciate any help in this regard. I have things as planned to do on the lab and when it is not sta...
- 2370 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi, was this resolved? can I know the solution please? Also, is there any restriction on the number of days or so to use this lab? I am getting 'Session ended' message even after I login again.RegardsDeepa
- 1 kudos
- 786 Views
- 2 replies
- 0 kudos
AWS Secrets Manager access
HiI am trying to establish a method of accessing secrets from AWS Secrets Manager and understand this can be done with boto as suggested from AWS.We have created all of the relevant IAM roles, instance profiles etc. Accessing S3 with this method is ...
- 786 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Alberto_Umana,Yes, the Role has the SecretsManagerReadWrite policy.Also within my furthet investigation I tried running it via Personal Cluster and it worked!Basically, 3 scenarios:- Shared Cluster with applied InstanceProfile - Secrets failing- ...
- 0 kudos
- 772 Views
- 1 replies
- 0 kudos
Communication between multiple region workspaces & metastore
Hi EveryoneWe currently have a workspace and metastore in the EastUS region, and we’re planning to set up another workspace and metastore in the Canada region. Additionally, we need to be able to access data from the Canada region within the US regio...
- 772 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @Dnirmania! Delta Sharing is ideal for read-only, cross-platform data access without duplication. Direct metastore connections offer low-latency access between Databricks workspaces under a unified governance model. Additionally, you may explor...
- 0 kudos
- 708 Views
- 1 replies
- 0 kudos
Query not returning result - no internet connection
Hi,New to DBricks. We have set up our new environment and imported some data to tables. However when we run a query against the table using a newly created warehouse we can see the query runs fine but throw the error: failed communicating with serve...
- 708 Views
- 1 replies
- 0 kudos
- 0 kudos
It sounds like there might be a network configuration issue or a client-side problem preventing your query results from appearing in the session, even though they're executing successfully on the server. Here are a few things you could check:Network ...
- 0 kudos
- 2388 Views
- 2 replies
- 2 kudos
Restrict a Workspace User from Creating/Managing Databricks Jobs
Hello Databricks team,I currently have a workspace user, and I want to disable their ability to create or manage Databricks jobs entirely. Specifically, I would like to prevent the user from accessing the "Create Job" option in the Databricks UI or v...
- 2388 Views
- 2 replies
- 2 kudos
- 2 kudos
Are there any updates on any of these internal feature requests? It's a pretty big failure in the permissions model that we can't prevent users from scheduling arbitrary workloads on compute they are able to access.
- 2 kudos
