cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
Mits
by • New Contributor II
  • 2574 Views
  • 4 replies
  • 3 kudos

Sending email alerts to non-databricks user

I am trying to send email alerts to a non databricks user. I am using Alerts feature available in SQL. Can someone help me with the steps.Do I first need to first add Notification Destination through Admin settings and then use this newly added desti...

Data Engineering
Reply
  • 2574 Views
  • 4 replies
  • 3 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Mitali Lad​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers yo...

  • 3 kudos
3 More Replies
francly
by • New Contributor II
  • 3651 Views
  • 5 replies
  • 3 kudos

Resolved! terraform create multiple db user

Hi, follow the example to create one user. It's working however I want to create multiple users, I have tried many ways but still cannot get it work, please share some idea.https://registry.terraform.io/providers/databricks/databricks/latest/docs/res...

Data Engineering
Reply
  • 3651 Views
  • 5 replies
  • 3 kudos
View Replies
Latest Reply
Natlab
New Contributor II
  • 3 kudos

What if I want to give User Name along with the email ID?I used below code but its not helping(code is not failing, but not adding user name)It seems this code line: "display_name = each.key" is not working. Pls suggest.  terraform {required_provider...

  • 3 kudos
4 More Replies
AndyAtINX
by • New Contributor III
  • 3070 Views
  • 4 replies
  • 1 kudos

Resolved! Error inviting user to workspace "Failed to add user: A user with email ... or username ... in different cases already exist in the account"

We have 3 workspaces - 1 old version in one AWS account, 2 latest versions in another.We are PAYG full edition, not using SSO.Our admins (existing DBX users in the `admins` group) can invite new users via the Admin Console from the 1 old and 1 new wo...

Data Engineering
Reply
  • 3070 Views
  • 4 replies
  • 1 kudos
View Replies
Latest Reply
Schneider-Elect
New Contributor II
  • 1 kudos

We are facing same issue, We are on azure. @AndyAtINX you mean if user exist in workspace with abc@gmail.com we should add the user in workspace2 with abc@gmail.com not ABC@GMAIL.COM. if this the case we tried this and its not working for us.

  • 1 kudos
3 More Replies
peterwishart
by • New Contributor III
  • 4166 Views
  • 4 replies
  • 0 kudos

Resolved! Programmatically updating the “run_as_user_name” parameter for jobs

I am trying to write a process that will programmatically update the “run_as_user_name” parameter for all jobs in an Azure Databricks workspace, using powershell to interact with the Jobs API. I have been trying to do this with a test job without suc...

Data Engineering
Reply
  • 4166 Views
  • 4 replies
  • 0 kudos
View Replies
Latest Reply
baubleglue
New Contributor II
  • 0 kudos

  Solution you've submitted is a solution for different topic (permission to run job, the job still runs as the user in run_as_user_name field). Here is an example of changing "run_as_user_name"Docs:https://docs.databricks.com/api/azure/workspace/job...

  • 0 kudos
3 More Replies
gg_047320_gg_94
by • New Contributor II
  • 7984 Views
  • 1 replies
  • 1 kudos

DLT Spark readstream fails on the source table which is overwritten

I am reading the source table which gets updated every day. It is usually append/merge with updates and is occasionally overwritten for other reasons. df = spark.readStream.schema(schema).format("delta").option("ignoreChanges", True).option('starting...

Data Engineering
Reply
  • 7984 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
Debayan
Databricks Employee
  • 1 kudos

Hi, Could you please confirm DLT and DBR versions? Also please tag @Debayan​ with your next response which will notify me, Thank you!

  • 1 kudos
Himanshu_90
by • New Contributor III
  • 5692 Views
  • 8 replies
  • 7 kudos

Databricks sql not able to evaluate expression current_user

Hi,I have a table as below:create table default.test_user(ID bigint NOT NULL GENERATED BY DEFAULT AS IDENTITY (START WITH 1 INCREMENT BY 1),usr1 varchar(255) NOT NULL,ts1 timestamp NOT NULL,usr2 varchar(255) NOT NULL,ts2 timestamp NOT NULL) USING Del...

Data Engineering
Reply
  • 5692 Views
  • 8 replies
  • 7 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 7 kudos

Hi @Himanshu Agrawal​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us...

  • 7 kudos
7 More Replies
maymay1993
by • New Contributor II
  • 1553 Views
  • 2 replies
  • 2 kudos

where can I can the user manual fo databricks?

Data Engineering
Reply
  • 1553 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @may may​ Your input matters! Help our community thrive by coming back and marking the most helpful and accurate answers. Together, we can make a difference!Regards

  • 2 kudos
1 More Replies
Ajay-Pandey
by • Esteemed Contributor III
  • 1267 Views
  • 1 replies
  • 6 kudos

Cluster policies now support limiting the max number of clusters per user can create Policy permissions allow you to set a max number of clusters per ...

Cluster policies now support limiting the max number of clusters per user can createPolicy permissions allow you to set a max number of clusters per user. This determines how many clusters a user can create using that policy. If the user exceeds the ...

image
Data Engineering
Reply
  • 1267 Views
  • 1 replies
  • 6 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 6 kudos

Thank you for sharing

  • 6 kudos
joshberry
by • New Contributor II
  • 2757 Views
  • 2 replies
  • 0 kudos

Resolved! Unable to add password to a user (with SSO enabled)

I am trying to add a non-SSO admin user to my account (not to a workspace). I have SSO backed off to Google for the majority of users.I can create the account OK, then go in and reset the password to something, but when I try and log in I get the err...

Data Engineering
Reply
  • 2757 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
joshberry
New Contributor II
  • 0 kudos

Ah, missed that bit of the docs. Thanks

  • 0 kudos
1 More Replies
data_explorer
by • New Contributor II
  • 1023 Views
  • 1 replies
  • 0 kudos

Is there anyway to execute grant and revoke statements to a user for an object based on a condition?

SELECT if((select count(*) from information_schema.table_privileges where grantee = 'samo@test.com' and table_schema='demo_schema' and table_catalog='demo_catalog')==1, (select count(*) from demo_catalog.demo_schema.demo_table), (select count(*) from...

Data Engineering
Reply
  • 1023 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi, GRANT and REVOKE are privileges on an securable object to a principal. And a principal is a user, service principal, or group known to the metastore. Principals can be granted privileges and may own securable objects.Also, you can use REVOKE ON S...

  • 0 kudos
whh99
by • New Contributor II
  • 1822 Views
  • 2 replies
  • 1 kudos

Given user id, what API can we use to find out which cluster the user is connected to?

I want to know the cluster that user is connected to in databricks. It would be great if we can also get the duration that the user is connected.

Data Engineering
Reply
  • 1822 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
daniel_sahal
Esteemed Contributor
  • 1 kudos

You can track activity logs by activating audit logs.I'm not sure which cloud provider you're using, but ex. for Azure you can find a manual here:https://learn.microsoft.com/en-us/azure/databricks/administration-guide/account-settings/audit-logs

  • 1 kudos
1 More Replies
jonathan-dufaul
by • Valued Contributor
  • 1988 Views
  • 2 replies
  • 2 kudos

Resolved! Does anyone have a single example of a graphframe with two+ types of vertices? (e.g. user and post, not user to user)

I have gone through about 75 pages and every single example has only relationships from one type of object to the same type of object. about 90% have the exact same example of "Alice Bob" "friends."Has anyone ever made a graphframe with two types of ...

Data Engineering
Reply
  • 1988 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
-werners-
Esteemed Contributor III
  • 2 kudos

I feel your pain,I once tried to use graphframes to flatten a complex tree, ended up using graphX (which is even worse to use but at least it is more flexible).So maybe take a look at graphX? Beware, it is terrible to use.I wonder what happened to m...

  • 2 kudos
1 More Replies
elgeo
by • Valued Contributor II
  • 1107 Views
  • 1 replies
  • 0 kudos

User logging in Databricks

Hello experts. Is there a way to see user logs as who is running which notebook or a specific query? I couldn't find anything in "Driver logs" (stdout, log4j). Just to specify that I have admin role in the workspace. Thank you in advance

Data Engineering
Reply
  • 1107 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
elgeo
Valued Contributor II
  • 0 kudos

Any update on this? Thank you

  • 0 kudos
jayallenmn
by • New Contributor III
  • 2408 Views
  • 2 replies
  • 3 kudos

Giving new user workspace access

Hey all,We have a new user we'd like to give access to our spark workspace. We invited the user to the workspace as an account admin. They click on the invite link and create a password and login. Once logged in they can see the workspace and can ...

Data Engineering
Reply
  • 2408 Views
  • 2 replies
  • 3 kudos
View Replies
Latest Reply
User16255483290
Contributor
  • 3 kudos

The new feature in data bricks is identity federation if identity federation is enabled then the users part of the data bricks account and the account admin can assign the users to the workspace. The account admins can add the users from account cons...

  • 3 kudos
1 More Replies
Trung
by • Contributor
  • 1374 Views
  • 2 replies
  • 1 kudos

Resolved! DataBricks best practice to manage resource correspond deleted user

currently I have some prblem about my DataBricks workspace when an user was deleted and it cause some issue:Applications or scripts that use the tokens generated by the user will no longer be able to access the Databricks APIJobs owned by the user wi...

Data Engineering
Reply
  • 1374 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
Trung
Contributor
  • 1 kudos

@Vivian Wilfred​ it really useful for my case, many thanks!

  • 1 kudos
1 More Replies
Labels