cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Discussions
Engage in dynamic conversations covering diverse topics within the Databricks Community. Explore discussions on data engineering, machine learning, and more. Join the conversation and expand your knowledge base with insights from experts and peers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Browse the Community

Community Discussions

Engage in vibrant discussions covering diverse learning topics within the Databricks Community. Expl...

4105 Posts

Activity in Discussions

chirag_nagar
by > • New Contributor
  • 544 Views
  • 2 replies
  • 1 kudos

Seeking Guidance on Migrating Informatica PowerCenter Workflows to Databricks using Lakebridge

Hi everyone,I hope you're doing well.I'm currently exploring options to migrate a significant number of Informatica PowerCenter workflows and mappings to Databricks. During my research, I came across Lakebridge, especially its integration with BladeB...

Data Engineering
Reply
  • 544 Views
  • 2 replies
  • 1 kudos
Reply
Latest Reply
Elyas
Visitor
  • 1 kudos

 I'm also curious. Looking forward for the answers with a same kind of questions. And also I could see there is a code repo shared by databricks for the lakebridge GitHub - databrickslabs/lakebridge: Accelerates migrations to Databricks by automating...

  • 1 kudos
1 More Replies
Cert-Team
by • Databricks Employee
  • 67918 Views
  • 185 replies
  • 16 kudos

Suspended Exam? Don't worry! Our support team can help!

Taking exams is so stressful already, and if your exam gets suspended it is extra stressful. Don't worry! Our support team is awesome and will help you get back on track. The most important step is to file a ticket in our Help Center. Be sure to inc...

Certifications
Reply
  • 67918 Views
  • 185 replies
  • 16 kudos
Reply
Latest Reply
mnsh22334466
Visitor
  • 16 kudos

Hi @Cert-Team I have faced the issue of exam suspension of Databricks Certified Data Engineer Associate exam.I show him my entire room everything.,and now in web assessor account my exam in in-progress. I did what was asked by proctor even it got sus...

  • 16 kudos
184 More Replies
IONA
by > • New Contributor II
  • 80 Views
  • 5 replies
  • 6 kudos

Getting data from the Spark query profiler

When you navigate to Compute > Select Cluster > Spark UI > JDBC/ODBC There you can see grids of Session stats and SQL stats. Is there any way to get this data in a query so that I can do some analysis? Thanks

Data Engineering
Reply
  • 80 Views
  • 5 replies
  • 6 kudos
Reply
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 6 kudos

 Hi @IONA ,As @BigRoux  correctly suggested there no native way to get stats from JDBC/ODBC Spark UI.1. You can try to use query history system table, but it has limited number of metrics %sql SELECT * FROM system.query.history 2. You can use /api/2....

  • 6 kudos
4 More Replies
Manila
by > • New Contributor
  • 4560 Views
  • 7 replies
  • 1 kudos

Resolved! How to access lab through partner databricks academy

Hi , I am able to login through partner-databricks-academy but not able to see any option for accessing the lab. please guide.

Databricks Free Trial Help
Reply
  • 4560 Views
  • 7 replies
  • 1 kudos
Reply
Latest Reply
sampmann2
New Contributor II
  • 1 kudos

Thank you @Advika I could see Lab after enrolling in an e-learning course. However, the lab was confined to the course. I am currently doing the "Data Ingestion with Lakeflow Connect" course that mentions a lab, but I don't see a way to access it. Ca...

  • 1 kudos
6 More Replies
Amogh
by > • New Contributor II
  • 927 Views
  • 3 replies
  • 2 kudos

To redeem the voucher again after cancelling the exam

Please advise.I wanted to take the Databricks Data Engineer Associate Exam, but by mistake I booked the Databricks Data Analyst Associate Exam with the voucher code.So, if I cancel the Databricks Data Analyst Associate Exam now will I be able to use ...

Certifications
Reply
  • 927 Views
  • 3 replies
  • 2 kudos
Reply
Latest Reply
Dhiyaneshwar
Visitor
  • 2 kudos

Hello, were you able to use the same voucher code for booking the exam?

  • 2 kudos
2 More Replies
Naveen_Sequeira
by > • New Contributor
  • 28 Views
  • 1 replies
  • 2 kudos

Need help in creating compute cluster

Hi everyone,I hope you’re doing well.I’m currently facing an issue while trying to create a cluster in Azure Databricks (Personal). Despite trying various approaches, I keep encountering the following error message:The VM size you are specifying is n...

Get Started Discussions
Reply
  • 28 Views
  • 1 replies
  • 2 kudos
Reply
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 2 kudos

Hi @Naveen_Sequeira ,If you increased quota and it didn't help then just try to use different region. In Azure, some machines are sometimes unavailable due to regional capacity issue. They should be available in another region though.You can try with...

  • 2 kudos
ck7007
by > • New Contributor II
  • 54 Views
  • 3 replies
  • 3 kudos

Advanced Technique

Reduced Monthly Databricks Bill from $47K to $12.7KThe Problem: We were scanning 2.3TB for queries needing only 8GB of data.Three Quick Wins1. Multi-dimensional Partitioning (30% savings)# Beforedf.write.partitionBy("date").parquet(path)# After-parti...

Data Engineering
Reply
  • 54 Views
  • 3 replies
  • 3 kudos
Reply
Latest Reply
BS_THE_ANALYST
Honored Contributor III
  • 3 kudos

@ck7007 no worries. I asked a question on the other thread: https://community.databricks.com/t5/data-engineering/cost/td-p/130078 , I'm not sure if you're classing this thread as the duplicate or the other one so I'll repost.I didn't see you mention ...

  • 3 kudos
2 More Replies
Pratikmsbsvm
by > • Contributor
  • 76 Views
  • 2 replies
  • 1 kudos

Read Files from Adobe and Push to Delta table ADLS Gen2

The Upstream is sending 2 files of different schema. The Storage Account has Private Endpoints. there is no public access.no public IP (NPIP) = yes.How to design using only Databricks :-1. Databricks API to read data file from Adobe and Push it to AD...

Pratikmsbsvm_0-1756741451588.png
Data Engineering
Reply
  • 76 Views
  • 2 replies
  • 1 kudos
Reply
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 1 kudos

Hi @Pratikmsbsvm ,Okay, since you’re going to use Databricks compute for data extraction and you wrote that your workspace is deployed with the secure connectivity cluster (NPIP) option enabled, you first need to make sure that you have a stable egre...

  • 1 kudos
1 More Replies
brian999
by > • Contributor
  • 3356 Views
  • 5 replies
  • 2 kudos

Resolved! Managing libraries in workflows with multiple tasks - need to configure a list of libs for all tasks

I have workflows with multiple tasks, each of which need 5 different libraries to run. When I have to update those libraries, I have to go in and make the update in each and every task. So for one workflow I have 20 different places where I have to g...

Data Engineering
Reply
  • 3356 Views
  • 5 replies
  • 2 kudos
Reply
Latest Reply
brian999
Contributor
  • 2 kudos

Actually I think I found most of a solution here in one of the replies: https://community.databricks.com/t5/administration-architecture/installing-libraries-on-job-clusters/m-p/37365/highlight/true#M245It seems like I only have to define libs for the...

  • 2 kudos
4 More Replies
rswarnkar5
by > • New Contributor II
  • 41 Views
  • 1 replies
  • 2 kudos

Resolved! Taking Databricks Certification exams through Authorized Test Centers

Hello there, I am aware that this has been asked here in past. But I would like to submit a suggestion for having a feature for the test takers from the test facilities.It would be nice to have Databricks Certification exams supported through Authori...

Certifications
Reply
  • 41 Views
  • 1 replies
  • 2 kudos
Reply
Latest Reply
szymon_dybczak
Esteemed Contributor III
  • 2 kudos

Very good idea @rswarnkar5 . I think it would definitely reduce the cases of users whose exam was interrupted just because they moved their head or glanced to the side for a split second

  • 2 kudos
NamrataHindujaG
by > • Visitor
  • 23 Views
  • 1 replies
  • 0 kudos

Namrata Hinduja Geneva - Issues with slow performance during merge operations

Hi everyone! I'm Namrata Hinduja Geneva, Switzerland and I'm working on a Delta Lake project in Databricks and facing issues with slow performance during merge operations on large datasets. I’ve optimized the table and used ZORDER, but the queries st...

Get Started Discussions
Reply
  • 23 Views
  • 1 replies
  • 0 kudos
Reply
Latest Reply
Khaja_Zaffer
Contributor
  • 0 kudos

Hello Namrata aka realnameHumble request, please stop spamming with HindujaG this is not right platform for you sir/madam. I hope you understand. 

  • 0 kudos
Cert-Team
by • Databricks Employee
  • 9640 Views
  • 1 replies
  • 1 kudos

Minimize the chances of your exam getting suspended - Tip #3

Being mindful of your behavior while taking an online proctored exam is another thing you can do to avoid an exam suspension. Here are some behavior tips to make your testing experience smoother: Take your exam in a private, quiet room with the door ...

Certifications
Reply
  • 9640 Views
  • 1 replies
  • 1 kudos
Reply
Latest Reply
rswarnkar5
New Contributor II
  • 1 kudos

Again to supplement why its good idea to have test centers: https://community.databricks.com/t5/certifications/taking-databricks-certification-exams-through-authorized-test/m-p/130451#M3581  

  • 1 kudos
der
by > • New Contributor III
  • 164 Views
  • 1 replies
  • 0 kudos

DBR 17.1 Spatial SQL Functions and Apache Sedona

I noticed in the DBR 17.1 release notes that ST geospatial functions are now in public preview - great news for us since this means native support in Databricks.https://docs.databricks.com/aws/en/release-notes/runtime/17.1#expanded-spatial-sql-expres...

Data Engineering
Reply
  • 164 Views
  • 1 replies
  • 0 kudos
Reply
Latest Reply
der
New Contributor III
  • 0 kudos

@mjohns @dbkent do you know more about Apache Sedona and DBR SQL Spatial Functions?

  • 0 kudos
gayatrikhatale
by > • New Contributor II
  • 95 Views
  • 4 replies
  • 1 kudos

Pass parameter value to sql query in databricks sql alert

Hi,I have created a sql query with the parameter.Now using that parameterized sql query in databricks legacy sql alert.And scheduling that databricks sql alert in databricks workflow/job using Sql task type.Now how should I pass the parameter value t...

Get Started Discussions
Reply
  • 95 Views
  • 4 replies
  • 1 kudos
Reply
Latest Reply
gayatrikhatale
New Contributor II
  • 1 kudos

Thanks for the clarification.

  • 1 kudos
3 More Replies
ilir_nuredini
by > • Honored Contributor
  • 1572 Views
  • 7 replies
  • 9 kudos

New Databricks Data Engineer Professional Exam!

Hello community!Databricks has released a new version of the Data Engineer Professional Certification.This version is currently in beta, and you can apply to take the certification exam for free and become one of the first to earn this privilege.Impo...

image.png
Certifications
Reply
  • 1572 Views
  • 7 replies
  • 9 kudos
Reply
Latest Reply
Nidhig
New Contributor III
  • 9 kudos

Hi ilir_nurediniWhat is the criteria for beta test attempt, I applied but got rejected.So just wanted to know how do Databricks select people.Thanks 

  • 9 kudos
6 More Replies
Load more

Featured Posts

View all