cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Lewis_Wong
by New Contributor II
  • 2047 Views
  • 4 replies
  • 2 kudos

Resolved! Adding tags to jobs from Tableau / Python (ODBC)

Hi all,We are using Azure Databricks.We would like to see if we can track usage of jobs initiated from Tableau /Python (via simba spark ODBC driver) One way we can think of is to add tag to the job. But we are not sure if we can add tags to job when ...

  • 2047 Views
  • 4 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

@Vidula Khanna​ We can add custom values to the job clusters as well. When you create the job task, click on edit cluster details which will take you to the cluster configuration where you can add the custom tags. In case of Tableau queries, it would...

  • 2 kudos
3 More Replies
Paradox_Parijat
by New Contributor III
  • 3997 Views
  • 7 replies
  • 4 kudos

Didn't receive Databricks badge

I just completed the Databricks Lakehouse Fundamentals exam, but while redirecting to credentials.databricks.com. I'm not able to find any badge provided to me for completion. Can you help me out? @Vidula Khanna​ 

  • 3997 Views
  • 7 replies
  • 4 kudos
Latest Reply
sher
Valued Contributor II
  • 4 kudos

check your mail. yesterday I have completed they shared me through registered email

  • 4 kudos
6 More Replies
Mafofola
by New Contributor II
  • 1283 Views
  • 3 replies
  • 0 kudos

Am using databricks trying to login​

Am using databricks trying to login​

  • 1283 Views
  • 3 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Ige Makanjuola​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us s...

  • 0 kudos
2 More Replies
ashish577
by New Contributor III
  • 6411 Views
  • 5 replies
  • 5 kudos

Is it not possible to create multiple tables on same location in unity catalog?

I have tried 2 tables with different catalogs, schemas, column names and it throws LOCATION_OVERLAP. In spark we can create as many tables as needed on the same location.

  • 6411 Views
  • 5 replies
  • 5 kudos
Latest Reply
Anonymous
Not applicable
  • 5 kudos

Hi @Ashish Singh​ I'm sorry you could not find a solution to your problem in the answers provided.Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.I suggest p...

  • 5 kudos
4 More Replies
Alyayman
by Contributor
  • 1884 Views
  • 3 replies
  • 3 kudos

Level of databricks spark certificate exam

I am willing to take spark certificate exam , i have solved the practice test and it was very good , but i am afraid that the level of exam is harder than the practice exam . So does it really harder ?

  • 1884 Views
  • 3 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Aly Ayman​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers you...

  • 3 kudos
2 More Replies
User16752244127
by Contributor
  • 2541 Views
  • 2 replies
  • 1 kudos

Resolved! DLT Security in Transit and in Rest

do you have docs that explain more specifics? is it end-to-end encrypted in transit? for in rest, is it just the encryption we get from e.g. S3?

  • 2541 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Frank Munz​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 1 kudos
1 More Replies
Elon
by New Contributor III
  • 2152 Views
  • 1 replies
  • 5 kudos

When are you gonna fix this site?

Unacceptable user experience for this forum: What markdown support?My colleagues agree the email verification looks sketchy.The website is SUPER slow. (10-56 seconds per page)Uses color red for email verified?!No feedback on repeated logins.The fonts...

ugly slow great 2023-03-15_10-09 databricks-smarting
  • 2152 Views
  • 1 replies
  • 5 kudos
Latest Reply
Anonymous
Not applicable
  • 5 kudos

Hi @Elon Musk​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedback wi...

  • 5 kudos
sanq
by New Contributor II
  • 4539 Views
  • 2 replies
  • 7 kudos

what formatter is used to format SQL cell in databricks

Databricks launched formatter Black which formats python cells, I can also see SQL cell getting formatted, but not sure which formatter is being used for SQL cell formatting. No clarity given on docs.

  • 4539 Views
  • 2 replies
  • 7 kudos
Latest Reply
Anonymous
Not applicable
  • 7 kudos

Hi @SANJAY qq​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 7 kudos
1 More Replies
Unilever
by New Contributor II
  • 1262 Views
  • 2 replies
  • 1 kudos

I would like to get rid of the error

the SPN we use for the mount points has access to the dataset in question, but for some reason I get this errorPlease find the attached screenshot for the error details.

Screenshot (4)
  • 1262 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @sai chandu palkapati​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from y...

  • 1 kudos
1 More Replies
bd
by New Contributor III
  • 4244 Views
  • 3 replies
  • 0 kudos

Resolved! Job aborted due to stage failure: ModuleNotFoundError

I'm getting this Failure Reason on a fairly simple streaming job. I'm running the job in a notebook. The notebook relies on a python module that I'm syncing to DBFS with `dbx`. Within the notebook generally, the module is available, i.e. `import mymo...

  • 4244 Views
  • 3 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Benjamin Dean​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...

  • 0 kudos
2 More Replies
Dayaa
by New Contributor II
  • 2573 Views
  • 3 replies
  • 4 kudos

Resolved! Load data into Azure SQL Database from Azure Databricks ( restricted table not a whole workspace tables)

Hi ,I want to share limited tables in my databricks workspace and users will connect to my databricks through Azure Data factory and will load data into Azure SQL. Is this possible using Delta Sharing? Or any other method or tool?

  • 2573 Views
  • 3 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @Dayananthan Marimuthu​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your...

  • 4 kudos
2 More Replies
Philearner
by New Contributor II
  • 2697 Views
  • 3 replies
  • 3 kudos

Unable to find input by typing input in the Multiselect Widget

In the AWS databricks widgets.multiselect, I'm unable to find input by typing input in the mulitselect bar. It was working before. Although I can find the inputs by scrolling down the list, it's annoying if the list is long.​​Here's my script:measlis...

databrick widget problem databrick widget problem 2
  • 2697 Views
  • 3 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Philip Teu​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 3 kudos
2 More Replies
Sas
by New Contributor II
  • 5100 Views
  • 3 replies
  • 4 kudos

Resolved! Confusion in string comparison

Hello expertI am new to spark. I am using same price of code but getting different resultsWhen i am using below piece of code, i am getting errorpy4j.Py4JException: Method or([class java.lang.String]) does not existdf.filter(F.col("state").isNull()  ...

  • 5100 Views
  • 3 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @Saswata Dutta​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedbac...

  • 4 kudos
2 More Replies
sagiatul
by New Contributor II
  • 5379 Views
  • 2 replies
  • 3 kudos

Databricks driver logs

I am running jobs on databricks clusters. When the cluster is running I am able to find the executor logs by going to Spark Cluster UI Master dropdown, selecting a worker and going through the stderr logs. However, once the job is finished and cluste...

image
  • 5379 Views
  • 2 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Atul Arora​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedback w...

  • 3 kudos
1 More Replies
saikrishna3390
by New Contributor II
  • 7154 Views
  • 2 replies
  • 2 kudos

How do I configure managed identity to databricks cluster and access azure storage using spark config

Partner want to use adf managed identity to connect to my databricks cluster and connect to my azure storage and copy the data from my azure storage to their azure storage storage

  • 7154 Views
  • 2 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @SAI PUSALA​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedback w...

  • 2 kudos
1 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels