cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Ramacloudworld
by New Contributor
  • 541 Views
  • 1 replies
  • 0 kudos

cross join issue generating surrogate keys in delta table

I used below code to populate target table, it is working as expected but expect for surrogatekey column. After I inserted dummy entry -1 and did run merge code, it is generating the numbers in surrogatekey column like this 1,3,5,7,9 odd numbers, it ...

  • 541 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @Ramacloudworld, It seems you’re encountering an issue with the surrogate key generation in your merge code. Let’s break it down and address the problem. Surrogate Keys: A surrogate key is a system-assigned unique value used to identify an ent...

  • 0 kudos
Eric76
by New Contributor
  • 231 Views
  • 1 replies
  • 0 kudos

Can the community version of databrick run model training examples?

Hi, Newcomer here. I am experimenting with the community version of databrick.I wanted to run the notebook example provided here https://community.cloud.databricks.com/?o=6085264701896358#notebook/2691200955149229It failed because it cannot import th...

  • 231 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @Eric76, Welcome to the Databricks community! Let’s address the issue you’re facing with importing data in your notebook. The example notebook you’re trying to run relies on a dataset located at /dbfs/databricks-datasets/wine-quality/winequalit...

  • 0 kudos
Phani1
by Valued Contributor
  • 436 Views
  • 2 replies
  • 0 kudos

Capture changes at the object level in Databricks

Could you please suggest how to capture changes at the object level in Databricks, such as notebooks changes, table DDL changes, view DDL , functions DDL, and workflows etc. changes? We would like to build a dashboard for changes.

  • 436 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hey there! Thanks a bunch for being part of our awesome community!  We love having you around and appreciate all your questions. Take a moment to check out the responses – you'll find some great info. Your input is valuable, so pick the best solution...

  • 0 kudos
1 More Replies
wellington
by New Contributor III
  • 395 Views
  • 2 replies
  • 0 kudos

Log notebook activities

Hi friends;I'm working on a project where we are 4 programmers. We are working in a single environment, using only the "Workspaces" folder. Each has its own user, which is managed by Azure AD.We had a peak in consumption on the 5th Feb. So I can see ...

  • 395 Views
  • 2 replies
  • 0 kudos
Latest Reply
wellington
New Contributor III
  • 0 kudos

Hi @Kaniz , thanks for your quick answer.There is no other way to monitor notebook runs. I ask this because adding tags to the cluster and workspace does not solve my problem, considering that everyone uses the same cluster and the same workspace.

  • 0 kudos
1 More Replies
AadityaBhatt
by New Contributor II
  • 713 Views
  • 4 replies
  • 1 kudos

[UNBOUND_SQL_PARAMETER] When running Placeholder query

I am using the databricks-sql-go library version 1.5.2. I am trying to run a query with placeholders of type '?'The query looks like params, args := databricksParams(values)sql := fmt.Sprintf(`SELECT COUNT(*) FROM %s.%s WHERE %s IN (%s)`, schema, tab...

  • 713 Views
  • 4 replies
  • 1 kudos
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Hey there! Thanks a bunch for being part of our awesome community!  We love having you around and appreciate all your questions. Take a moment to check out the responses – you'll find some great info. Your input is valuable, so pick the best solution...

  • 1 kudos
3 More Replies
Dlt
by New Contributor III
  • 1267 Views
  • 5 replies
  • 0 kudos

Running an exe file in databricks

Hello I have an executable file which i want to host and run from databricks.  is this possible in databricks using DBFS ?If NOT what are the other ways to it in databricks ? 

  • 1267 Views
  • 5 replies
  • 0 kudos
Latest Reply
BR_DatabricksAI
Contributor
  • 0 kudos

Hello, I don't have much information on what kind of executables you would like to run in databricks however, I can think of two solutions : Solution 1: Deploy your code in azure container registry as an image and use the endpoint in data bricks. Sol...

  • 0 kudos
4 More Replies
ChristianRRL
by Contributor
  • 2265 Views
  • 5 replies
  • 3 kudos

Resolved! DLT Job Clusters: Continuous vs Triggered Cluster Start Times

Hi there,I'm curious if anyone is able to definitively help me answer how DLT Job Clusters operate/run.For example, the following is my baseline understanding of DLT Job Clusters. If I run a Triggered DLT Pipeline (e.g. daily) the job cluster takes m...

  • 2265 Views
  • 5 replies
  • 3 kudos
Latest Reply
Kaniz
Community Manager
  • 3 kudos

Hey there! Thanks a bunch for being part of our awesome community!  We love having you around and appreciate all your questions. Take a moment to check out the responses – you'll find some great info. Your input is valuable, so pick the best solution...

  • 3 kudos
4 More Replies
Lucidity
by New Contributor II
  • 3343 Views
  • 3 replies
  • 1 kudos

Resolved! Deny assignment modification to allow attach/detach of disks in azure databricks

Our application does storage autoscaling on Azure. We would like to deploy our solution with Azure databricks. But even though the service principal associated with our application has the necessary roles and permissions to attach/detach a disk from ...

  • 3343 Views
  • 3 replies
  • 1 kudos
Latest Reply
Lucidity
New Contributor II
  • 1 kudos

Thank you for your replyIs there any way databricks provides to bypass the deny assignment for specific apps? I noticed in the deny assignment unity-catalog-access-connector has been provided exlusion under the excludePrincipals section. is there a w...

  • 1 kudos
2 More Replies
Phani1
by Valued Contributor
  • 1380 Views
  • 2 replies
  • 0 kudos

Masking techniques for more PII columns

Hi Databricks Team,We would appreciate it if you could inform us about the situations when Column-Masking, Row-Level Filtering, and Attributed-Based Masking should be utilized, as well as the recommended technique for handling large data volumes cont...

  • 1380 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...

  • 0 kudos
1 More Replies
Wycliff
by New Contributor II
  • 624 Views
  • 2 replies
  • 0 kudos

JWT Encoding error while using Azure secret key

My secret value in Azure key vault is like below.private_key="""-----BEGIN RSA PRIVATE KEY-----********-----END RSA PRIVATE KEY-----"""Running this command in Databricks notebook - jwt.encode(claim_set,private_key,algorithm='RS256')While using the ab...

  • 624 Views
  • 2 replies
  • 0 kudos
Latest Reply
Wycliff
New Contributor II
  • 0 kudos

Thanks much for your troubleshooting methods.Validated the secret scopes, accessing secrets. These looks fine.Key format - I feel problem is with the key format only. As of now I'm awaiting on Azure subscription access. But I printed the secret value...

  • 0 kudos
1 More Replies
Sweetness
by New Contributor II
  • 836 Views
  • 3 replies
  • 0 kudos

Cannot create a repo because the parent path does not exist

I tried following this docWork With Large Monorepos With Sparse Checkout Support in Databricks Repos | Databricks BlogWhen I hook it up to my repos using Azure DevOps Services and check mark Sparse checkout mode, I pass in a subdirectory in my Cone p...

  • 836 Views
  • 3 replies
  • 0 kudos
Latest Reply
Sweetness
New Contributor II
  • 0 kudos

This is Azure Databricks

  • 0 kudos
2 More Replies
mathijs-fish
by New Contributor III
  • 1195 Views
  • 3 replies
  • 3 kudos

Resolved! "with open" not working with Shared Access Cluster on mounted location

Hi All,For an application that we are building, we need a encoding detector/utf-8 enforcer. For this, we used the python library chardet in combination with "with open". We open a file from a mounted adls location (we use a legacy hive-metastore)When...

mathijsfish_1-1701785425743.png mathijsfish_2-1701785466668.png
Get Started Discussions
glob
Mount
os
with open
  • 1195 Views
  • 3 replies
  • 3 kudos
Latest Reply
Ayushi_Suthar
Honored Contributor
  • 3 kudos

Hi @mathijs-fish thank you for sharing the solution, it will help us to update our records and documentation, enabling us to assist other customers more effectively in similar cases.

  • 3 kudos
2 More Replies
Lambda
by New Contributor
  • 328 Views
  • 1 replies
  • 0 kudos

What is the REST API payload for "not sending alert when alert is back to normal"?

Greeting,I am using REST API to create & schedule sql alert by default these alerts will send notifications when they are back to normal which I don't want, In the UI I have the option to uncheck the box (shown in the picture) but I can't find any do...

Lambda_0-1702306187563.png
  • 328 Views
  • 1 replies
  • 0 kudos
Latest Reply
Yeshwanth
Valued Contributor II
  • 0 kudos

@Lambda Did you refer this documentation: https://docs.databricks.com/api/workspace/alerts/create

  • 0 kudos
Phani1
by Valued Contributor
  • 351 Views
  • 1 replies
  • 0 kudos

Spark VCore for Databrick vCPU

Hi Team,What is the equivalent Spark VCore for Databrick vCPU? , like for below example for DS3 v2 , vCPU=4 and RAM = 14.00 GiB, would like to know equivalent Spark VCore for DS3 v2 as in  Azure Databricks Pricing | Microsoft AzureRegards,Phanindra

  • 351 Views
  • 1 replies
  • 0 kudos
Latest Reply
Yeshwanth
Valued Contributor II
  • 0 kudos

@Phani1 In Databricks, the vCPU count is equivalent to the number of Spark vCores. This means that if the DS3 v2 instance has 4 vCPU, it would also have 4 Spark vCores. Please note that the Spark VCore count is based on the vCPU count of the underlyi...

  • 0 kudos
AtomicBoy99
by New Contributor
  • 355 Views
  • 1 replies
  • 0 kudos

Can't enter 'edit mode' using shortcut

Hi,With the recently added AI Assistant to databricks notebooks, I'm having issues entering the 'edit mode' of a notebook cell. Previously, I could simply press the 'Enter' key in order to do this but this no longer works.Is anyone else having the sa...

  • 355 Views
  • 1 replies
  • 0 kudos
Latest Reply
Yeshwanth
Valued Contributor II
  • 0 kudos

@AtomicBoy99, it seems like the fix is in place. Can you confirm if it is working well now?

  • 0 kudos
Labels
Top Kudoed Authors