- 1646 Views
- 2 replies
- 1 kudos
Databricks Audit Logs Analysis
Please share the attributes related to the audit logsHow is the audit logs can be utilized by cyber security team? What are the insights into the audit logs and how we can maintain the compliance? What are the non-compliance items can be identified f...
- 1646 Views
- 2 replies
- 1 kudos
- 1 kudos
The audit logs will only get the information of the events and actions being performed by users, service principals in the workspace, so there is no compliance actions being tracked itself.The Service name is a subgroup of items you want to check, so...
- 1 kudos
- 1889 Views
- 1 replies
- 0 kudos
not able to configure databricks with aws
I am trying to set up Databricks with my AWS account but I am facing some issues, I followed the steps to create a new Databricks workspace using the AWS CloudFormation template or Databricks setup guide, But the CloudFormation stack fails every time...
- 1889 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @dipin_wantoo! Can you please check the CloudFormation Stack Events for the exact error? That should help identify why the stack is failing. If you're looking to get started with Databricks as an individual user, the Express Setup is a simple w...
- 0 kudos
- 2300 Views
- 3 replies
- 1 kudos
Library installation failed for library due to user error for pypi
Hi!I get the below error when a cluster job starts up and tries to install a Python .whl file. (Which is hosted on an Azure Artefact feed, though this seems more like a problem of trying to read from a disk/network storage). The failure is seemingly ...
- 2300 Views
- 3 replies
- 1 kudos
- 1 kudos
Thanks both - I think the problem is that this library installation is called when creating a new Job & Task via the rest endpoint. Where the libraires are specified in the .json file. So short version, don't think I can 'get at' the pip install call...
- 1 kudos
- 2712 Views
- 2 replies
- 1 kudos
Resolved! Why am I seeing NAT Gateway in the cost? Serverless Compute.
I have Azure Databricks premium subscription. I am running the Python interactive Notebooks in the Databricks Wokrspace using Serverless compute since the last few days. Today I received an alter in my email saying the monthly Billing already crossed...
- 2712 Views
- 2 replies
- 1 kudos
- 1 kudos
Thanks @szymon_dybczak I deleted the Workspace and the NAT Gatway service got deleted from the vnet. I created a simple single node cluster to run my code.
- 1 kudos
- 1608 Views
- 1 replies
- 0 kudos
Resolved! Can I add another user to Free edition
Is it possible to add another user to Free edition ?I am wanting to test what they can see when they connect as a restricted user i.e. only granted Browse on 1 catalogThanks
- 1608 Views
- 1 replies
- 0 kudos
- 0 kudos
Apologies. Yes you can - I was in the wrong console.
- 0 kudos
- 4572 Views
- 7 replies
- 0 kudos
Failing PowerBi connection with data Bricks via SQL warehouse
I'm encountering an 'Invalid credentials' error (Session ID: 4601-b5a6-0daf792752a2, Region: us) when connecting Power BI to an Azure Databricks SQL Warehouse using an SPN. The SPN has CAN MANAGE access at SQL warehouse, admin rights at account and...
- 4572 Views
- 7 replies
- 0 kudos
- 0 kudos
Hello sai, sorry for the experience I usually available at desk during EMEA time zone. Apologies for the delay. Can you please try this?In "Advanced settings" of the data source within the gateway configuration and set the "Connection Encryption se...
- 0 kudos
- 2548 Views
- 1 replies
- 0 kudos
Looking for insights on enabling Databricks Automatic Provisioning
We currently have a SCIM provisioning connector set up to synchronize identities from Entra ID to Unity Catalog.We’re now considering enabling Databricks Automatic Provisioning but want to fully understand the potential impact on our environment befo...
- 2548 Views
- 1 replies
- 0 kudos
- 0 kudos
TranslateHello Xaveri.Good day!Here are few links related to databricks provisioninghttps://docs.databricks.com/aws/en/admin/users-groups/scim/aadhttps://www.databricks.com/blog/announcing-automatic-identity-management-azure-databricksBut do let me k...
- 0 kudos
- 3562 Views
- 4 replies
- 4 kudos
Privileged Identity Management for Databricks with Microsoft Entra ID
Privileged Identity Management (PIM) can be used to secure access to critical Databricks roles with Just-in-Time (JIT) access. This approach helps organizations enforce time-bound permissions, approval workflows, and centralized auditing for sensitiv...
- 3562 Views
- 4 replies
- 4 kudos
- 1836 Views
- 2 replies
- 0 kudos
Delete unassigned catalogs
Hi everybody,due to some not-so-optimal Infrastructure as code experiments with terraform I ended up a lot (triple digit) of catalogs in a metastore that are not assigned to any workspace and that i want to delete.Unfortunately, there is no way to ev...
- 1836 Views
- 2 replies
- 0 kudos
- 0 kudos
Yeah, I see those catalogs and i know that I could reattach and delete them. As i have around 100 those catalogs it would be nice to iterate through them by getting a list, e.g. using the cli or the rest API. And then force delete them , as described...
- 0 kudos
- 1441 Views
- 1 replies
- 1 kudos
VS Code - ipynb vs py execution - spark issue
Databricks Connect works inside VS Code notebook but the same code fails in a standalone script withValueError: default auth: cannot configure default credentialsI’m developing locally with **Databricks Connect 16.1.6** and VS Code.Inside a Jupyter n...
- 1441 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Sisi ,I think what's happening here is when you debug with option "Debug current file with Databricks Connect" then VS Code is using Databricks extension, which automatically handles authentication and sets up proper configuration.The regular Pyt...
- 1 kudos
- 3482 Views
- 6 replies
- 2 kudos
Out of memory error when installing environment dependencies of UC Python UDF
Hi,I've created a small UC Python UDF to test whether it works with custom dependencies (new PP feature), and every time I'm getting OOM errors with this message: [UDF_ENVIRONMENT_USER_ERROR.OUT_OF_MEMORY] Failed to install UDF dependencies for <cata...
- 3482 Views
- 6 replies
- 2 kudos
- 2 kudos
I tried with cluster, spent some couple of hours to load some libraries but unable to do. may be someone else can help you on this.
- 2 kudos
- 2654 Views
- 5 replies
- 4 kudos
Resolved! Metastore deletion issues
Good afternoon, I have an issue with my metastore in North Europe.All my workspaces got detached:If I go to Databricks console, I can see the metastore in North Europe I created.However, when I select the metastore in North Europe, I get the followin...
- 2654 Views
- 5 replies
- 4 kudos
- 4 kudos
I solved the issue by deleting all the asignments before deleting the metastore.1. Access to Databricks CLI and authenticate 2. List metastores>> databricks account metastores list 3. List wotrkspaces and check assignments>> databricks account worksp...
- 4 kudos
- 1812 Views
- 1 replies
- 1 kudos
Drop schema or catalog using cascade function
Hello In Databricks (non-Unity Catalog), I have two schemas (schema_a and schema_b) that both use the same root location in DBFS or external storage like ADLS.Example:abfss://container@storage_account.dfs.core.windows.net/data/project/schema_aabfss:/...
- 1812 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello @EjB For the given example, here is the response:Will DROP SCHEMA schema_a CASCADE remove or affect tables in schema_b?No, unless:1. The tables in schema_a are managed tables, AND2. Tables in schema_b store their data physically inside /schema_...
- 1 kudos
- 2399 Views
- 3 replies
- 0 kudos
[INTERNAL_ERROR] Query could not be scheduled: HTTP Response code: 503. Please try again later
We have a databricks job configured to run a dbt project. The dbt cli compute cluster being used is serverless with a serverless sql warehouse. We encountered this error during a run. SQLSTATE: XX000Any idea why this occurred?
- 2399 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi,We are experiencing same recurring HTTP errors throughout the day when executing queries on Databricks. The specific error message we receive is: "[INTERNAL_ERROR] Query could not be scheduled: HTTP Response code: 503. Please try again later. SQLS...
- 0 kudos
- 2336 Views
- 2 replies
- 3 kudos
Resolved! System tables performance optimization
Hi AllAre there any Databricks lab projects or GitHub repositories that leverage system tables to provide dashboards or code for monitoring, and more importantly, for optimizing workflows and clusters based on usage?
- 2336 Views
- 2 replies
- 3 kudos
- 3 kudos
+1 to @szymon_dybczak. I would also add the dashboards for DB SQL Warehouse monitoring.
- 3 kudos
