- 3002 Views
- 2 replies
- 1 kudos
Resolved! Difference between username and account_id
I have a web app that can read files from a person's cloud-based drive (e.g., OneDrive, Google Drive, Dropbox). The app gets access to the files using OAuth2. The app only ever has access to the files for that user. Part of the configuration requir...
- 3002 Views
- 2 replies
- 1 kudos
- 1 kudos
The provided links were helpful. The take-away* usernames are "globally" unique to an individual; the username is the person's email.* a username can be associated with up to 50 accounts; account_ids track the resources available to the user.This cl...
- 1 kudos
- 1212 Views
- 0 replies
- 0 kudos
cross join issue generating surrogate keys in delta table
I used below code to populate target table, it is working as expected but expect for surrogatekey column. After I inserted dummy entry -1 and did run merge code, it is generating the numbers in surrogatekey column like this 1,3,5,7,9 odd numbers, it ...
- 1212 Views
- 0 replies
- 0 kudos
- 1904 Views
- 1 replies
- 0 kudos
Resolved! Unable to create a Managed table in Unity Catalog defaul location
We have setup the metastore with Manged Identity and when trying to create a managed table in the default location I am hitting below error. The storage is ADLS Gen2. AbfsRestOperationException: Operation failed: "This request is not authorized to pe...
- 1904 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Kris2 , I completely understand your hesitation and appreciate your approach to seeking guidance! This error generally means that the cluster has connectivity to Unity Catalog configured storage location and it is not authorized to access storag...
- 0 kudos
- 649 Views
- 0 replies
- 1 kudos
Error Signing Up for Databricks Community Edition
@Retired_mod I've been trying to sign up for Databricks Community Edition using different email addresses over the past 24 hours, but I keep getting the error message: "An error has occurred. Please try again later." Can anyone help?Tags: #Databricks...
- 649 Views
- 0 replies
- 1 kudos
- 1597 Views
- 0 replies
- 0 kudos
Unity Catalog: Databricks *Specific* Features
Good day,Deceptively simple question, are there any "Databricks only" specific features that Unity Catalog offers? I understand that generally speaking enabling UC offers some of the following:Data Discovery and LineageAuditing and MonitoringAccess C...
- 1597 Views
- 0 replies
- 0 kudos
- 3332 Views
- 2 replies
- 0 kudos
Problem login in
Hello allI´m new in this platform, I sign up, validated my email, create my password everything is fine and when I try to log in and start a message came upI create a new password but same happen again! But it works a few times, I think like 3 times....
- 3332 Views
- 2 replies
- 0 kudos
- 0 kudos
Can you confirm the username was created all lower case? Login is case sensitive so you need to make sure the username is set same exact as you add it in the console or workspace
- 0 kudos
- 2151 Views
- 3 replies
- 1 kudos
Resolved! DataBricks Certification Exam Got Suspended. Require Immediate support
Hello @Cert-Team @Certificate Team,Request Id# 00432042I encountered a pathetic experience while attempting my Databricks Certified Data Engineer Professional certification exam. This is a completely unethical process to harass the examinee and lose...
- 2151 Views
- 3 replies
- 1 kudos
- 1 kudos
@shrutigupta12 I'm sorry to see you had difficulty with your video feed. Glad to see the proctor was able to help resolve the issue and you were able to complete your exam.
- 1 kudos
- 5483 Views
- 3 replies
- 3 kudos
Resolved! DLT Job Clusters: Continuous vs Triggered Cluster Start Times
Hi there,I'm curious if anyone is able to definitively help me answer how DLT Job Clusters operate/run.For example, the following is my baseline understanding of DLT Job Clusters. If I run a Triggered DLT Pipeline (e.g. daily) the job cluster takes m...
- 5483 Views
- 3 replies
- 3 kudos
- 3 kudos
Ideally one would expect clusters used for DLT pipeline to terminate after the pipeline execution has finished. However, while running in `development` environment, you'll notice it doesn't terminate on its own, whereas in `production` it terminates ...
- 3 kudos
- 2259 Views
- 1 replies
- 0 kudos
Databricks Java - Create Jar in Java 11
I am trying to a run simple print java program which is not working and getting compilation version issues though i changed the environment variable points to java 11. Can you please help me ? Can we create java with spark session and execute as a ja...
- 2259 Views
- 1 replies
- 0 kudos
- 0 kudos
@Databricks_Java You can run command like this: spark-submit --class com.test.Main example.jarand make sure to check the java version and match with the DBR compatibility
- 0 kudos
- 1293 Views
- 1 replies
- 0 kudos
Disable personal compute with the Databricks API or UI
For a production environment, I want to disable the personal compute policy, because I do not want that all users can create personal compute clusters in production. Unfortunately, I am not able to access the account console, so I want to revoke perm...
- 1293 Views
- 1 replies
- 0 kudos
- 0 kudos
@mathijs-fish You need to be admin to disable a policy.
- 0 kudos
- 1455 Views
- 3 replies
- 0 kudos
Autoloader file latency
Hi Team,I would like to understand if there is a metadata table for the autoloader in Databricks that captures information about file arrival and processing.The reason we are experiencing data issues is because our table A receives hundreds of files ...
- 1455 Views
- 3 replies
- 0 kudos
- 0 kudos
Check with cloud_files_state() API You can find examples here https://docs.databricks.com/en/ingestion/auto-loader/production.html#querying-files-discovered-by-auto-loader
- 0 kudos
- 3521 Views
- 2 replies
- 0 kudos
Resolved! spark context in databricks
Hi @all,In Azure Databricks,I am using structured streaming for each batch functionality, in one of the functions I am creating tempview with pyspark dataframe (*Not GlobalTempView) and trying to access the same temp view by using spark.sql functiona...
- 3521 Views
- 2 replies
- 0 kudos
- 0 kudos
Do you face this issue without spark streaming as well? Also, could you share a minimal repo code preferably without streaming?
- 0 kudos
- 1508 Views
- 1 replies
- 0 kudos
Generate Excel for a SQL query
Greetings,I am using a Java Spring boot application that is supposed to respond with an excel based on request. My current approach involves reading data using jdbc drivers, storing them in appropriate data structures, writing them to an excel which ...
- 1508 Views
- 1 replies
- 0 kudos
- 0 kudos
Thanks for putting this together @Retired_mod ,I see that this approach will help to generate an excel after receiving the data from data bricks in the form of resultSet which has to be parsed.I believe this approach is the appropriate way to generat...
- 0 kudos
- 977 Views
- 0 replies
- 0 kudos
Data Masking issue
Hi Team,After successfully creating a group and populating it with members, I am facing the problem of the is_member() function always returning False, despite the user being part of the group. This issue is specific to my sandbox Databricks Account....
- 977 Views
- 0 replies
- 0 kudos
- 1153 Views
- 1 replies
- 0 kudos
Data masking best practices
Hi Team,Could you please suggest any best practices/blogs on implementing data masking, row level ,column level ,access control, role-based access control (RBAC), and attribute-based access control (ABAC)? Regards.Phanindra
- 1153 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi, Can you check if this document answers your question: https://www.databricks.com/blog/2020/11/20/enforcing-column-level-encryption-and-avoiding-data-duplication-with-pii.html
- 0 kudos
