cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

rams
by Contributor
  • 1914 Views
  • 4 replies
  • 4 kudos

Rollback error - Configuring Databricks lakehouse platform with AWS account

I have logged in databricks account and while creating the workspace i have chosen quickstart approach to configure the databricks with AWS. During the quickstart process the databricks page will redirect to aws cloudformation stack page where the ac...

  • 1914 Views
  • 4 replies
  • 4 kudos
Latest Reply
Kaniz
Community Manager
  • 4 kudos

Hi @rams shonu​ ​, We haven’t heard from you since the last response from @Pat Sienkiewicz​​, and I was checking back to see if my suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to o...

  • 4 kudos
3 More Replies
Serhii
by Contributor
  • 1354 Views
  • 2 replies
  • 0 kudos

Resolved! Horovod Databricks Job - custom module not found error

We have used the following example to successfully create a distributed deep learning training notebook https://www.databricks.com/blog/2022/09/07/accelerating-your-deep-learning-pytorch-lightning-databricks.html that works as expected.We now want to...

  • 1354 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @Sergii Ivakhno​ ​, We haven’t heard from you since the last response from me​ ​, and I was checking back to see if my suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to others.Als...

  • 0 kudos
1 More Replies
Sufyan
by New Contributor II
  • 905 Views
  • 3 replies
  • 2 kudos

Resolved! When and where would I receive my Certificate after passing the exam?

I have passed the Databricks Certified Data Analyst Associate exam on 29 October 2022. I have not received the corticate yet.

  • 905 Views
  • 3 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Sufyan Shafique​ We are really sorry for the delays.Just a friendly- follow-up, have you got your certification and badge? If yes, please mark the answer as best.Thanks and Regards

  • 2 kudos
2 More Replies
jefft
by New Contributor III
  • 1642 Views
  • 3 replies
  • 1 kudos

Databricks integration with DocumentDB

Hi Everyone,I was wondering if anyone here has any experience or tips reading data from AWS DocumentDB. I am working on this using the MongoDB connector. For DocumentDB we also need to work with the required creds issued as a .pem file by AWS. Th...

  • 1642 Views
  • 3 replies
  • 1 kudos
Latest Reply
jefft
New Contributor III
  • 1 kudos

Hi @Kaniz Fatma​ ,Thank you so much for your response. Your suggestions were helpful. As per the AWS documentation, DocumentDB is MongoDB compatible. "With Amazon DocumentDB, you can run the same application code and use the same drivers and tools th...

  • 1 kudos
2 More Replies
StanleyTang
by New Contributor III
  • 1536 Views
  • 4 replies
  • 4 kudos

How to run SQL queries from services when data migrated from SQL server to data lake?

Currently our service provides an API to serve the purchase records. The purchase records are stored in SQL database. To simplify, when users want to get their recent purchase records, they make an API call. The API call will run a SQL query on the D...

  • 1536 Views
  • 4 replies
  • 4 kudos
Latest Reply
Kaniz
Community Manager
  • 4 kudos

Hi @Stanley Tang​​​, We haven’t heard from you since the last response from @Debayan Mukherjee​ , and I was checking back to see if his suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful...

  • 4 kudos
3 More Replies
Sanjujain
by New Contributor II
  • 821 Views
  • 2 replies
  • 1 kudos

Resolved! I have created account but forgot to check community edition option while signing up. How this can be solved now.

I have created account but forgot to check community edition option while signing up. How this can be solved now.

  • 821 Views
  • 2 replies
  • 1 kudos
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Hi @serg-v (Customer)​​, We haven’t heard from you since the last response from me​ ​, and I was checking back to see if my suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to others.A...

  • 1 kudos
1 More Replies
Sajid1
by Contributor
  • 3731 Views
  • 6 replies
  • 1 kudos
  • 3731 Views
  • 6 replies
  • 1 kudos
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Hi @Sajid Thavalengal Rahiman​, We haven’t heard from you since the last response from @Laurent Léturgez​ ​ ​, and I was checking back to see if his suggestions helped you.Or else, If you have any solution, please share it with the community, as it c...

  • 1 kudos
5 More Replies
marcus1
by New Contributor III
  • 2193 Views
  • 9 replies
  • 2 kudos

SQL Warehouse connections in us-east-2

I have 2 very similarly configured workspaces, one in us-west-2 and one in us-east-2.Both got configured by default with a "Starter Warehouse".The one in us-west-2 I can reach via the internet using python databricks-sql-connector, but the one in us-...

  • 2193 Views
  • 9 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Marcus Simonsen​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...

  • 2 kudos
8 More Replies
Sandy21
by New Contributor III
  • 837 Views
  • 2 replies
  • 1 kudos

Resolved! Databricks Jobs

Consider User A has deployed the job to prod. User B has scheduled the job through an external orchestration tool.User C has got the owner privileges from User A. Whose email id would be displayed while running the databricks job?

  • 837 Views
  • 2 replies
  • 1 kudos
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Hi @Sandeep Negi​ , We haven’t heard from you since the last response from me​ ​, and I was checking back to see if my suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to others.Also, ...

  • 1 kudos
1 More Replies
Dinu2
by New Contributor III
  • 1711 Views
  • 2 replies
  • 4 kudos

Cassandra connection from ADB

Hi , Could anyone please help to know the steps for connecting Cassandra from Azure Databricks? I have followed the steps in https://learn.microsoft.com/en-us/azure/databricks/_static/notebooks/azure/cassandra-azure.html But I am getting below error....

  • 1711 Views
  • 2 replies
  • 4 kudos
Latest Reply
Kaniz
Community Manager
  • 4 kudos

Hi @Dinu Sukumara​, We haven’t heard from you on the last response from me, and I was checking back to see if my suggestions helped you. Or else, If you have any solution, please do share that with the community as it can be helpful to others.Also, P...

  • 4 kudos
1 More Replies
ShibashishRoy
by New Contributor
  • 1106 Views
  • 1 replies
  • 0 kudos

unable to find LoginModule class: org.apache.kafka.common.security.plain.PlainLoginModule

In databricks I am trying to read data from a protected kafka topic using pyspark.I am getting an error "unable to find LoginModule class: org.apache.kafka.common.security.plain.PlainLoginModule".

  • 1106 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kakfa_Topic
New Contributor II
  • 0 kudos

Did you able to resolve it?

  • 0 kudos
Gim
by Contributor
  • 52633 Views
  • 5 replies
  • 9 kudos

Best practice for logging in Databricks notebooks?

What is the best practice for logging in Databricks notebooks? I have a bunch of notebooks that run in parallel through a workflow. I would like to keep track of everything that happens such as errors coming from a stream. I would like these logs to ...

  • 52633 Views
  • 5 replies
  • 9 kudos
Latest Reply
Kaniz
Community Manager
  • 9 kudos

Hi @Gimwell Young​ , We haven’t heard from you on the last response from @karthik p​​, @Debayan Mukherjee​ and @Hubert Dudek​  and I was checking back to see if their suggestions helped you. Or else, If you have any solution, please do share that wit...

  • 9 kudos
4 More Replies
164079
by Contributor II
  • 2007 Views
  • 4 replies
  • 0 kudos

Resolved! authentication is not configured for provider

Hi team, I started getting this message lately when trying add some new config or change my workspace with terraform :Error: cannot create global init script: authentication is not configured for provider. Please check https://registry.terraform.io/p...

image
  • 2007 Views
  • 4 replies
  • 0 kudos
Latest Reply
Vivian_Wilfred
Honored Contributor
  • 0 kudos

Hi @Avi Edri​ looks like you are using a provider that is authenticated to the Accounts console (https://accounts.cloud.databricks.com) to create a global init script within the workspace. Can you try authentication with host and PAT token? Follow th...

  • 0 kudos
3 More Replies
Data_Engineer3
by Contributor II
  • 3402 Views
  • 2 replies
  • 7 kudos

Move folder from dbfs location to user workspace directory in azure databricks

I need to move group of files(python or scala file from)or folder from dbfs location to user workspace directory in azure databricks to do testing on file.Its verify difficult to upload each file one by one into the user workspace directory, so is it...

  • 3402 Views
  • 2 replies
  • 7 kudos
Latest Reply
Kaniz
Community Manager
  • 7 kudos

Hi @KARTHICK N​, We haven’t heard from you on the last response from @Werner Stinckens​​, and I was checking back to see if his suggestions helped you. Or else, If you have any solution, please do share that with the community as it can be helpful to...

  • 7 kudos
1 More Replies
ferbystudy
by New Contributor III
  • 2075 Views
  • 4 replies
  • 3 kudos

Resolved! Can´t read a simple .CSV from a blob

Guys, I am using "Databricks Community" to study. I put some files in a Blob, granted all access but I have no ideia why DB is not reading. Please see the code below and thanks for helping! thanks!

csf
  • 2075 Views
  • 4 replies
  • 3 kudos
Latest Reply
Kaniz
Community Manager
  • 3 kudos

Hi @Fernando Rezende​, Thank you for sharing the solution with us.It would mean a lot if you could select the "Best Answer" to help others find the correct answer faster.This makes that answer appear right after the question, so it's easier to find w...

  • 3 kudos
3 More Replies
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels