- 687 Views
- 3 replies
- 1 kudos
Identifying workload in azure and AWS
we are looking for some python codes that can helps us, we need to have an overview of all Databricks workspaces, their owner names, and mainly the runtime versions that they use, in every Azure and AWS subscriptions that we manage. Can someone pleas...
- 687 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @Saurabh_kanoje, There are two complementary approaches to get an overview of all your Databricks workspaces, their owners, and the runtime versions in use across Azure and AWS. I will walk through both. APPROACH 1: SYSTEM TABLES (RECOMMENDED, NO ...
- 1 kudos
- 382 Views
- 2 replies
- 0 kudos
Databricks Workspace ACL Enforcement
Databricks Workspace ACL Enforcement – How to Prevent Users from Creating Objects Outside Team Folder and Attaching to Shared Clusters?BackgroundI am configuring workspace-level access control in Databricks to restrict Data Engineers (DE group) to op...
- 382 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @APJESK, The two behaviors you are observing are both by design in how Databricks workspace ACLs work. Let me walk through each one and then cover what you can do to tighten governance. ISSUE 1: USERS CAN CREATE NOTEBOOKS IN THEIR HOME FOLDER Ever...
- 0 kudos
- 1114 Views
- 2 replies
- 2 kudos
Best Practice for Sharing AI/BI Dashboards across Workspaces in the same Account
Hello everyone,I’m looking for the most efficient way to share dashboards between two workspaces (Workspace A and Workspace B) within the same Databricks account.[Current Setup]Account: Single account with two workspaces (A and B).Data Governance: Bo...
- 1114 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Seunghyun, This is a common architecture question, and there are several approaches depending on your requirements around freshness, governance, and operational overhead. Let me address each of your questions directly and then recommend an overal...
- 2 kudos
- 972 Views
- 3 replies
- 1 kudos
Installing libraries on job clusters using tasks dependencies is not reliable in case of repairs
Hello,Following the suggestion on this thread, for job clusters we install the libraries only on the first task of the workflow, which are then made available to the subsequent tasks.However, this method is not reliable in the case of run repairs: th...
- 972 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @aliz, This is a common pattern to run into when using a dedicated "setup" task for library installation on shared job clusters. The core issue is that repair runs provision a fresh job cluster and only re-execute the failed (or selected) tasks, s...
- 1 kudos
- 612 Views
- 1 replies
- 0 kudos
Automatic Identity Management
Hello,In the context of reviewing our company's databricks structure and migrating legacy workspaces to Unity Catalog enabled ones, we're stuck with a few questions regarding enabling the automatic identity management feature.We currently provision D...
- 612 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @andrefilipemm, These are important questions when planning a migration to identity federation and Unity Catalog. Let me address each one. BACKGROUND: HOW IDENTITY FEDERATION WORKS When you enable identity federation on a workspace, identity manag...
- 0 kudos
- 889 Views
- 2 replies
- 1 kudos
Implement & Test DR Plan in AWS databricks
Can you share detailed steps or document for DR setup.Example Consider Workspace A running in us-east-1, and planning to setup DR on us-west-2So what are the steps and Task I need to create on AWS us-west-2.Please share in detail steps that required ...
- 889 Views
- 2 replies
- 1 kudos
- 1 kudos
@APJESK I have seen this pattern before. disaster recovery planning for Databricks on AWS is a critical topic and one that Databricks has solid documentation and tooling around. Let me walk you through a comprehensive approach to implementing and tes...
- 1 kudos
- 852 Views
- 7 replies
- 3 kudos
Non-admin users hit browser error opening Jobs → [Job] → Tasks tab (admins OK)
Hi Databricks Community,We’re seeing what appears to be a Jobs UI bug affecting non-admin users only.SummaryWhen a non-admin user opens a job and clicks the Tasks tab, the UI fails and shows an error modal. Admin users can access the same job’s Tasks...
- 852 Views
- 7 replies
- 3 kudos
- 3 kudos
Hi @bdocken, Great that the product team will be looking at your workspace directly -- that is the fastest path to a definitive fix for the redirect loop and React error #185 you are seeing. While you wait for that investigation to complete, here are...
- 3 kudos
- 923 Views
- 4 replies
- 0 kudos
Unable to add users to Azure AD–synced Databricks group via SCIM (InternalError)
Hi everyone,I’m trying to add users to an existing Databricks group programmatically using the Databricks SDK (SCIM PATCH), but I consistently hit an internal error and can’t tell if this is expected behavior or a limitation.My main goal is adding us...
- 923 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @discuss_darende, @KartikBhatnagar is on the right track in the replies above. Let me expand on this with the full picture. The "InternalError" you are seeing when trying to add users to an Azure AD-synced group via SCIM is expected behavior. Grou...
- 0 kudos
- 1943 Views
- 3 replies
- 0 kudos
How setup jdbc connection to snowflake
Hi, im on free Databricks tier account and need help understanding how create jdbc connection to snowflake, as I'm getting error:Catalog > (+) > Create a connection: Connection name= jdbc_snowflake Connection type= JDBC [next] Url= jdbc//org-ac...
- 1943 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @emanueol, The error you are seeing ("Connection type JDBC requires environment settings") is because you selected "JDBC" as the connection type, which is a generic connector intended for databases that do not have a dedicated connection type in D...
- 0 kudos
- 332 Views
- 1 replies
- 0 kudos
Payment with Netsuite Customer Payment Portal
I was using a trial account and credit was finished. Now i have done my first month and my card has been rejected,At the moment I received a warning email where i should pay at this platform named as Netsuite Customer Payment Portal but I can't login...
- 332 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @vino2000, Thanks for reaching out. I understand the frustration -- your trial credits ran out, your card was declined, and now you have received an email pointing you to the NetSuite Customer Payment Portal but you cannot log in or reset your pas...
- 0 kudos
- 639 Views
- 3 replies
- 1 kudos
External connectivity to VNET Injected & SCC enabled workspace
Hi All,We have a Databricks installation that is in our internal azure vnets (using vnet injection). And secure cluster connectivity is enabled as well. We have couple of external partners who wants to connect to our workspaces using APIs and JDBC. W...
- 639 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi mat723, That approach is sound and is actually a fairly common pattern. Standing up a public-facing proxy/gateway that is locked down to only the partner's IP ranges, with the backend connecting to the Databricks workspace over Private Link, gives...
- 1 kudos
- 1343 Views
- 6 replies
- 4 kudos
Resolved! Databricks Apps Processes and Pain Points
I'm really interested in learning the processes of the creation and use of Databricks Apps in production that people are using right now. Anybody who has done so, I'd love it if you could provide info on what some of your pain points during these pro...
- 1343 Views
- 6 replies
- 4 kudos
- 4 kudos
Hi @odrobek, Great question -- Databricks Apps is still maturing, and sharing real-world workflows helps everyone. I have been working with Apps across several projects, so here is a rundown of what the production development cycle typically looks li...
- 4 kudos
- 797 Views
- 2 replies
- 1 kudos
Resolved! Streamlit alternative
Hi all,I have a simple app that contains an editable grid and displays some graphs. The Streamlit app is slow, and end users need a faster solution. We are using sql warehouse (serverless)The app will show data of the existing delta tables and user w...
- 797 Views
- 2 replies
- 1 kudos
- 1 kudos
I have had good results using Dash for apps that need charts and interactive UIs. It’s a solid option if you’re moving beyond Streamlit. Why Dash works well for your use case: Charts: Plotly (built into Dash) gives you interactive charts and good ...
- 1 kudos
- 5867 Views
- 6 replies
- 3 kudos
Resolved! How to create Storage Credential using Service Principal [Azure]
As the document indicates, An Azure Databricks access connector is a first-party Azure resource that lets you connect managed identities to an Azure Databricks account. You must have the Contributor role or higher on the access connector resource in ...
- 5867 Views
- 6 replies
- 3 kudos
- 3 kudos
Late reply.The databricks managed Access Connector is used by Databricks to manage its own "managed storage account" ( the one with the randomized characters in the name), which is used "for its own business", when spinning up and managing compute cl...
- 3 kudos
- 1182 Views
- 2 replies
- 1 kudos
Using Terraform to GRANT SELECT ON ANY FILE securable
I have a use case where service principals will read .csv files from Azure Storage Account and create views from them. This used to work in our legacy environment but we are currently migrating to Unity Catalog and when we tested our existing jobs we...
- 1182 Views
- 2 replies
- 1 kudos
- 1 kudos
How about... (not tried it myself)resource "databricks_sql_permissions" "grant_select_any_file" {any_file = true privilege_assignments { principal = "your-role" privileges = ["SELECT"] }}
- 1 kudos
