cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Prabha
by New Contributor II
  • 4016 Views
  • 4 replies
  • 0 kudos

DataBricks_dataengineer_result

Hi Team,I've successfully passed the Databricks Data Engineer Associate Certified exam on 30th October 2022. but still have not received the certificate.Please find the reference below.  I have registered the exam with the email id vprabhakaran1987@...

image
  • 4016 Views
  • 4 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Prabhakaran velusamy​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from y...

  • 0 kudos
3 More Replies
rupiniravi
by New Contributor II
  • 2013 Views
  • 2 replies
  • 1 kudos

Screenshot_20221031-222242_Chrome

Hi Databricks Team,​I​ have completed my databricks Certified data engineer Associate exam on Oct 30 but not received badge or certificate yet. Raised case alo but no response..could someone helpUserid: rupinisarguru128@gmail.com ​​​

  • 2013 Views
  • 2 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Rupini Ravichandran​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from yo...

  • 1 kudos
1 More Replies
Own
by Contributor
  • 1463 Views
  • 2 replies
  • 0 kudos
  • 1463 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Shubham Sharma​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...

  • 0 kudos
1 More Replies
Renga87
by New Contributor II
  • 3196 Views
  • 4 replies
  • 2 kudos

Clarification on Voucher code

Is voucher code is mandatory to register exam "Databricks Associate developer for Apache spark 3.0" If so how to get it. Note : Created account in databricks portal recently and username is ksrenga87@gmail.com

  • 3196 Views
  • 4 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Rengaraja Sundararaj​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from y...

  • 2 kudos
3 More Replies
Sinead
by New Contributor II
  • 3205 Views
  • 2 replies
  • 0 kudos

Resolved! Why do I keep getting locked out of my Community Edition Account?

I use Databricks Community Edition for college work and every time I try to log in, I find that I am locked out of my account, despite using the correct username and password. I get the message "You entered an invalid email or password, or your works...

  • 3205 Views
  • 2 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Sinead Walsh​ Thank you for reaching out, and we’re sorry to hear about this log-in issue! We have this Community Edition login troubleshooting post on Community. Please take a look, and follow the troubleshooting steps. If the steps do not resol...

  • 0 kudos
1 More Replies
Cano
by New Contributor III
  • 5594 Views
  • 4 replies
  • 2 kudos

SQL warehouse failing to start ( Please check network connectivity from the data plane to the control plane )

Hi, My SQL warehouse is failing to start with the following error message:Details for the latest failure: Error: [id: InstanceId(i-01b84b6705ff09104), status: INSTANCE_INITIALIZING, workerEnvId:WorkerEnvId(workerenv-3023557811934763-c8cef827-a038-455...

  • 5594 Views
  • 4 replies
  • 2 kudos
Latest Reply
Debayan
Databricks Employee
  • 2 kudos

Hi, There is a line in the attached logs as below:[Bootstrap Event] Can reach ohio.cloud.databricks.com: [FAILED][Bootstrap Event] DNS output for databricks-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com: Server: 10.187.0.2Address: 10.187.0.2#5...

  • 2 kudos
3 More Replies
dheeraj2444
by New Contributor II
  • 4048 Views
  • 3 replies
  • 0 kudos

I am trying to write a data frame to Kafka topic with Avro schema for key and value using a schema registry URL. The to_avro function is not writing t...

I am trying to write a data frame to Kafka topic with Avro schema for key and value using a schema registry URL. The to_avro function is not writing to the topic and throwing an exception with code 40403 something. Is there an alternate way to do thi...

  • 4048 Views
  • 3 replies
  • 0 kudos
Latest Reply
Debayan
Databricks Employee
  • 0 kudos

Hi,Could you please refer to https://github.com/confluentinc/kafka-connect-elasticsearch/issues/59 and let us know if this helps.

  • 0 kudos
2 More Replies
ackerman_chris
by New Contributor III
  • 3745 Views
  • 4 replies
  • 0 kudos

Resolved! Databricks Lakehouse Fundamentals Badge Not Found

Hello, I've successfully completed the Databricks Lakehouse Fundamentals and am looking to find where the badge is.I found this post here. But I haven't received email on my completion from <service.accredible.email@databricks.com> yet. I successfull...

  • 3745 Views
  • 4 replies
  • 0 kudos
Latest Reply
ackerman_chris
New Contributor III
  • 0 kudos

Thank You all for the great responses, I eventually received the Badge, it took around 30+ minutes to receive, but I finally did get the Email notification. I will mark this post as resolved

  • 0 kudos
3 More Replies
KrishZ
by Contributor
  • 21498 Views
  • 3 replies
  • 3 kudos

[Pyspark.Pandas] PicklingError: Could not serialize object (this error is happening only for large datasets)

Context: I am using pyspark.pandas in a Databricks jupyter notebook and doing some text manipulation within the dataframe..pyspark.pandas is the Pandas API on Spark and can be used exactly the same as usual PandasError: PicklingError: Could not seria...

  • 21498 Views
  • 3 replies
  • 3 kudos
Latest Reply
ryojikn
New Contributor III
  • 3 kudos

@Krishna Zanwar​ , i'm receiving the same error.​For me, the behavior is when trying to broadcast a random forest (sklearn 1.2.0) recently loaded from mlflow, and using Pandas UDF to predict a model.​However, the same code works perfectly on Spark 2....

  • 3 kudos
2 More Replies
Anuj_Diggibyte
by Databricks MVP
  • 4499 Views
  • 2 replies
  • 0 kudos

How to overwrite partition in DLT pipeline ?

I am trying to replicate my existing spark pipeline in DLT. I am not able to achieve desired result using DLT . Current pipeline : source set up : CSV file ingested in bronze using SCP frequency : monthly bronze dir : /cntdlt/bronze/emp/year=2022 /...

  • 4499 Views
  • 2 replies
  • 0 kudos
Latest Reply
kfoster
Contributor
  • 0 kudos

What I have observed, @dlt.table with a spark.read or dlt.read will create the table in mode=overwrite@dlt.table with a spark.readStream or dlt.readStream will append new datato get the update, use the CDC: Change data capture with Delta Live Tables ...

  • 0 kudos
1 More Replies
SIRIGIRI
by Contributor
  • 1373 Views
  • 1 replies
  • 1 kudos

sharikrishna26.medium.com

Difference between “ And ‘ in Spark Dataframe APIYou must tell your compiler that you want to represent a string inside a string using a different symbol for the inner string.Here is an example.“ Name = “HARI” “The above is wrong. Why? Because the in...

  • 1373 Views
  • 1 replies
  • 1 kudos
Latest Reply
sher
Valued Contributor II
  • 1 kudos

thanks for sharing

  • 1 kudos
Raghu101
by New Contributor III
  • 7852 Views
  • 6 replies
  • 3 kudos

How to Call Oracle Stored Procedures from Databricks?

How to Call Oracle Stored Procedures from Databricks?

  • 7852 Views
  • 6 replies
  • 3 kudos
Latest Reply
sher
Valued Contributor II
  • 3 kudos

https://datathirst.net/blog/2018/10/12/executing-sql-server-stored-procedures-on-databricks-pyspark/try this link. this may help you

  • 3 kudos
5 More Replies
A_Jabbar
by New Contributor
  • 3873 Views
  • 2 replies
  • 2 kudos

Resolved! I am unable to create databricks community edition account!!!!!!

This is what I am doing,enter all the details on page 1 click on the Getting stated with community edition, after verification, I get the following error

Error Message on the second page of Registration
  • 3873 Views
  • 2 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Abdul Jabbar​ Thank you for reaching out, and we’re sorry to hear about this log-in issue! We have this Community Edition login troubleshooting post on Community. Please take a look, and follow the troubleshooting steps. If the steps do not resol...

  • 2 kudos
1 More Replies
KKo
by Contributor III
  • 2854 Views
  • 2 replies
  • 5 kudos

Read and write to XMLA from Databricks notebook

I am trying to process power bi dataset partition refresh from Azure Databricks, using XMLA endpoint. I have power bi premium capacity and read/write enabled. Tried few approaches found in google did not work with one or the other reason. If any of y...

  • 2854 Views
  • 2 replies
  • 5 kudos
Latest Reply
Anonymous
Not applicable
  • 5 kudos

Hi @Kris Koirala​  Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question first. Or else bricksters will get back to you soon. Thanks.

  • 5 kudos
1 More Replies
KVNARK
by Honored Contributor II
  • 4765 Views
  • 4 replies
  • 6 kudos

Resolved! best practices for SQL DB authentication from data bricks

I would like to know the best practices to authenticate SQL db from databricks/python. More interested to hear about some token based DB authentication methods other than credential based(username/password)

  • 4765 Views
  • 4 replies
  • 6 kudos
Latest Reply
Vivian_Wilfred
Databricks Employee
  • 6 kudos

@KVNARK .​ Have you checked on the PAT token for authentication? https://docs.databricks.com/sql/api/authentication.html

  • 6 kudos
3 More Replies
Labels