- 7451 Views
- 11 replies
- 3 kudos
Resolved! Valid Workspace Conf keys
HiI'm trying to automate the configuration of Admin Settings of our Databricks Workspace using Terraform. However identifying the correct config keys is very difficult.Databricks exposes a Workspace Conf API (Enable/disable features | Workspace Conf ...
- 7451 Views
- 11 replies
- 3 kudos
- 3 kudos
something that may be of interest. Not a substitute to official documentation:https://github.com/databricks/terraform-provider-databricks/issues/3365
- 3 kudos
- 467 Views
- 1 replies
- 0 kudos
Serverless and private connectivity - unable to create managed table
Hi,I am trying to setup Private connectivity for my serverless compute to my managed storage which is at the catalog level.I created the NCC, endpoints, external location and credentials with the required access. My storage account public network acc...
- 467 Views
- 1 replies
- 0 kudos
- 0 kudos
Try to do an nslooup to your storage from a notebook.# From your Databricks cluster, test DNS resolution%sh nslookup yourstorageaccount.blob.core.windows.net # Should resolve to private IP, not public IP # Check if storage account allows traffic f...
- 0 kudos
- 581 Views
- 3 replies
- 1 kudos
Resolved! Monitoring pool usage
Hi All,I am working on creating a dashboard for databricks instance pools. I am capturing maximum usage and scheduled a jpb to capture the info every 15 mins and I aggregate to see if any point the max usage is greater then 85% of capacity, Is there ...
- 581 Views
- 3 replies
- 1 kudos
- 1 kudos
The ai_forecast function did the magic and it have the forecast on pool usage that predicted for next 30 days
- 1 kudos
- 10909 Views
- 18 replies
- 19 kudos
how do you disable serverless interactive compute for all users
I don't want users using serverless interactive compute for their jobs. how do i disable it for everyone or for specific users
- 10909 Views
- 18 replies
- 19 kudos
- 19 kudos
Hi Team, Any update on this? We'd like to stop serverless compute access as well
- 19 kudos
- 474 Views
- 3 replies
- 2 kudos
Support for Unity Catalog External Data in Fabric OneLake
Hi community!We have set up a Fabric Link with our Dynamics, and want to attach the data in Unity Catalog using the External Data connector.But it doesn't look like Databricks support other than the default dls endpoints against Azure.Is there any wa...
- 474 Views
- 3 replies
- 2 kudos
- 2 kudos
Thanks @szymon_dybczak , do you know if support is on the roadmap? Since the current supported way of doing this with credential passthrough on the Compute is deprecated.Regards Marius
- 2 kudos
- 1102 Views
- 2 replies
- 3 kudos
Resolved! Can't create a new table from uploaded file.
I've just started using the Community Edition through the AWS marketplace and I'm trying to set up tables to share with a customer. I've managed to create 3 of the tables, but when uploading a small version of the fourth file, I'm having problems. T...
- 1102 Views
- 2 replies
- 3 kudos
- 3 kudos
Thank you, Lou.By loading manually, I found the error that wasn't being displayed in the UI. Once I took care of this, everything loaded just fine.
- 3 kudos
- 445 Views
- 3 replies
- 0 kudos
Usage of Databricks apps or UI driven approach to create & maintain Databricks infrastructure
Hi all,the CI/CD based process to create & maintain Databricks infrastructure (UC securables, Metastore securables, Workspace securables) is resulting into high time to market in our case. So, we are planning to make it UI driven, as on, create a Da...
- 445 Views
- 3 replies
- 0 kudos
- 0 kudos
UI Driven approach is definetly a bad idea for deployment. I have seen most organisation using terraform or biceps for deployment.Why UI-Driven Infrastructure is Wrong1. No version control or audit trail2. Configuration Drift & Inconsistency3. No Dis...
- 0 kudos
- 2460 Views
- 13 replies
- 2 kudos
Resolved! Which API to use to list groups in which a given user is a member
Is there an API that can be used to list groups in which a given user is a member? Specifically, I’d be interested in account (not workspace) groups.It seems there used to be a workspace-level list-parents API referred to in the answers to this quest...
- 2460 Views
- 13 replies
- 2 kudos
- 2 kudos
there will be 2 system tables soon - users, groups. Probably, that makes life easy. I have already requested the Databricks RAS meant for my customer to enabled these for our control plane
- 2 kudos
- 534 Views
- 4 replies
- 2 kudos
Resolved! Unable to get external groups from list group details API
Hi Team, I see that automatic group sync is enabled in our azure databricks by default but I am not able to get the groups using list groups endpoint. What is the right way to get it. I've an automated process to bring certain account groups to works...
- 534 Views
- 4 replies
- 2 kudos
- 2 kudos
I checked it and I can confirm that it worked. I enabled automatic identity management, then I've added EntraId group to the workspace and using above endpoint I was able to get information about that group.
- 2 kudos
- 450 Views
- 2 replies
- 0 kudos
Is there a way to see Job/Workflow in the lineage of a table/view
Hi allOne frequent question I get from the data users is - how often a view is refreshed? ((users are given access to views not tables))I was thinking to guide them to leverage lineage for the same. However, I noticed the lineage tab in the catalog e...
- 450 Views
- 2 replies
- 0 kudos
- 0 kudos
@Vidhi_Khaitan the user wants to know the frequency of the schedule, not whether the view/table is fresh...
- 0 kudos
- 421 Views
- 1 replies
- 0 kudos
Job Runs Permissions for Runs Submit API via ADF
I wanted to assign group permissions to Job runs in Azure Databricks which are triggered from ADF via Runs Submit API.How can we assign Job Runs Permissions in Azure Databricks for ADF triggers by Runs Submit API?
- 421 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @hajojko ,Good Day!I believe this is what you can try for your use case - https://docs.databricks.com/api/workspace/jobs_21/submit#access_control_list
- 0 kudos
- 583 Views
- 1 replies
- 0 kudos
Databricks compute failed to start due to EC2 security group id change in AWS default VPC
I have been using Databricks on AWS for almost 2 years. Recently, I cannot start up Databricks compute. "databricks_error_message": "The VM launch request to AWS failed, please check your configuration. [details] InvalidGroup.NotFound: The se...
- 583 Views
- 1 replies
- 0 kudos
- 0 kudos
hey @sg-vtc You need to register a new network configuration in Databricks that points to the updated VPC and security group. Here’s how:- Go to the Databricks Account Console (not the workspace UI). (Click in workspaces dropbox and Manage Account)- ...
- 0 kudos
- 308 Views
- 1 replies
- 0 kudos
Costs incurred by systems table usage
Doc about system tables says "The billable usage and pricing tables are free to use. Tables in Public Preview are also free to use during the preview but could incur a charge in the future."https://learn.microsoft.com/en-us/azure/databricks/admin/sys...
- 308 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @alsetr ,The billable usage and pricing system tables are the clearly marked as free to use.Tables in Public Preview are also free for now.For the rest, there’s no mention of a direct cost, but you’ll still be billed for the compute (DBUs) used wh...
- 0 kudos
- 1148 Views
- 3 replies
- 1 kudos
Using logged in user's identity in Databricks Apps
Hi Databricks Community, I recently started using Datbricks apps where I list some schemas and tables in the UI.What I explicitly want to do is only show the schemas and tables on which user actually have permission. Currently the databricks apps wou...
- 1148 Views
- 3 replies
- 1 kudos
- 1 kudos
https://docs.databricks.com/aws/en/dev-tools/databricks-apps/auth#retrieve-user-authorization-credentials : they launched this recently to support user identity in databricks apps
- 1 kudos
- 291 Views
- 1 replies
- 0 kudos
Custom hostname for Azure Databricks workspace
For a client it's required that all applications use a subdomain of an internal hostname. Does anyone have a suggestion on how to achieve this? Options could be in the line of CNAME or Reverse Proxy, but couldn't get that to work yet.Does Azure Datab...
- 291 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @gt-ferdyh, Azure Databricks doesn’t natively support custom subdomains. However, to make it accessible via an internal subdomain, you can:Enable Azure Private Link for your Databricks workspace.In your internal DNS, create an A record or CNAME th...
- 0 kudos
