cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

PabloSeoaneF
by New Contributor III
  • 5716 Views
  • 5 replies
  • 15 kudos

Databricks cluster's problems at start

Hello,I've spent some time trying to initialize a couple of clusters into an Azure Databricks private environment.Apparently, without reason they've been causing these two failures: With the advice of retrying in some minutes. Looking for information...

image.png image image
  • 5716 Views
  • 5 replies
  • 15 kudos
Latest Reply
PabloSeoaneF
New Contributor III
  • 15 kudos

Hello both! I attach here the 2 error details:Bootstrap timeout:HelpBootstrap Timeout. Please try again later.Instance bootstrap failed command: BootstrapTimeoutFailure message: Bootstrap script took too long and timeout. Please try again later.VM ex...

  • 15 kudos
4 More Replies
carlosancassani
by New Contributor III
  • 2438 Views
  • 3 replies
  • 5 kudos

Error: Credential size is more than configured size limit. As a result credential passthrough won't work for this notebook run.

I get this error when trying to execute parallel slave notebook from a Pyspark "master notebook".note 1: I use same class, functions, cluster, credential for another use case of parallel notebook in the same databricks instance and it works fine.note...

image
  • 2438 Views
  • 3 replies
  • 5 kudos
Latest Reply
Anonymous
Not applicable
  • 5 kudos

Hi @carlosancassani​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...

  • 5 kudos
2 More Replies
Mohit_Jain19
by New Contributor III
  • 3469 Views
  • 8 replies
  • 4 kudos

I passed the Databricks Machine Learning Associate Exam on 2022-11-18. I have not yet received the certificate/Badge. please help Thanks,Mohit Jain

I passed the Databricks Machine Learning Associate Exam on 2022-11-18. I have not yet received the certificate/Badge.please helpThanks,Mohit Jain

  • 3469 Views
  • 8 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @Mohit Jain​ Hope all is well! Just wanted to check in that did you receive your badge? If yes then please mark it as the best answer. Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 4 kudos
7 More Replies
Kush
by New Contributor III
  • 3414 Views
  • 3 replies
  • 4 kudos

Unable to find the Databricks Lakehouse Fundamentals Badge (Ceritifcate ID: E-E04YDV)

Hi, I completed my Fundamentals of the Databricks Lakehouse Platform Accreditation on 11/6/2022 but I have not received my badge yet. I have received a certificate which looks kind of fake. Can you please let me know by when will I receive my badge?

  • 3414 Views
  • 3 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @Kushagra Sen​ Hope all is fine!Just a friendly follow-up, have you received your badge? If yes, kindly mark the answer as best and if it hasn't been resolved yet then do let us know for further assistance. Thanks and Regards

  • 4 kudos
2 More Replies
ellaine
by New Contributor III
  • 7483 Views
  • 12 replies
  • 8 kudos

Google Ad Manager Reporting API Authentication via Databricks

Hello - I was wondering whether anyone has had any experience fetching data through GAM's Reporting API via Databricks.   The Reporting API requires installation of the "googleads" library, as well as a googleads.yaml file. I was able to find some do...

  • 7483 Views
  • 12 replies
  • 8 kudos
Latest Reply
Anonymous
Not applicable
  • 8 kudos

Hi @Ellaine Ho​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 8 kudos
11 More Replies
Anonymous
by Not applicable
  • 1955 Views
  • 2 replies
  • 16 kudos

Resolved! How to access data files for Databricks Workspace directly through Azure Blob Storage

Hi everyone,This is the first time I used Azure to deploy a Databricks (before I quite familiar use AWS for deploying Databricks). I want to view Databricks Workspace file directly from Azure portal but always don't have a permission.Could you give m...

Image
  • 1955 Views
  • 2 replies
  • 16 kudos
Latest Reply
Unforgiven
Valued Contributor III
  • 16 kudos

@Jensen Ackles​ read document as link below :https://docs.databricks.com/external-data/azure-storage.htmli dont know whats steps ur done, hope this can help u in this case

  • 16 kudos
1 More Replies
Tarique
by New Contributor II
  • 4104 Views
  • 4 replies
  • 3 kudos

Pyspark CSV Incorrect Count

B1123451020-502,"","{""m"": {""difference"": 60}}","","","",2022-02-12T15:40:00.783Z B1456741975-266,"","{""m"": {""difference"": 60}}","","","",2022-02-04T17:03:59.566Z B1789753479-460,"","",",","","",2022-02-18T14:46:57.332Z B1456741977-123,"","{""...

  • 4104 Views
  • 4 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Tarique Anwer​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...

  • 3 kudos
3 More Replies
baz
by New Contributor II
  • 1234 Views
  • 0 replies
  • 1 kudos

access to my account

access to my accountI can't access to my accountI am getting password reset link in my email but The password reset doesn't work.Invalid email address or passwordNote: Emails/usernames are case-sensitive

  • 1234 Views
  • 0 replies
  • 1 kudos
ravikanthranjit
by New Contributor III
  • 4298 Views
  • 6 replies
  • 14 kudos

Vacuum on external tables that we mount on ADLS

Want to know the best process of removal of files on ADLS after Optimize and Vacuum Dry run is completed

  • 4298 Views
  • 6 replies
  • 14 kudos
Latest Reply
Anonymous
Not applicable
  • 14 kudos

Hi @Ravikanth Narayanabhatla​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear fr...

  • 14 kudos
5 More Replies
Matt101122
by Contributor II
  • 3181 Views
  • 4 replies
  • 4 kudos

Resolved! Why are calls to /api/2.1/jobs/create taking 30-60 seconds to complete?

We are having an issue with calls to /api/2.1/jobs/create taking between 30 and 60 seconds to complete. Calls to /api/2.1/jobs/list complete as expected in about 6 seconds so the issue seems to be only with the job create API. This seems to only be h...

  • 3181 Views
  • 4 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @Matthew Dalesio​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...

  • 4 kudos
3 More Replies
ebg
by New Contributor III
  • 4135 Views
  • 3 replies
  • 10 kudos

I have an Azure storage account, and I need to list the containers on that storage account using scala from databricks notebook

I tried to run azure-cli on databricks (i am using Credential passthrough and my account needs MFA ) and do the following code:az storage container list --account-name "account_name" --auth-mode login --query "[].name" --output tsvHowever, it outputs...

  • 4135 Views
  • 3 replies
  • 10 kudos
Latest Reply
Anonymous
Not applicable
  • 10 kudos

Hi @elias bou ghosn​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...

  • 10 kudos
2 More Replies
AJDJ
by New Contributor III
  • 3153 Views
  • 2 replies
  • 6 kudos

Cost as per the Databricks demo

Hi there,I came across this Databricks demo from the below link. https://youtu.be/BqB7YQ1-KKcKindly Fastforward to time 16:30 or 16:45 of the video and watch few mins of the video related to cost. My understanding is the data is in the lake and datab...

  • 3153 Views
  • 2 replies
  • 6 kudos
Latest Reply
Anonymous
Not applicable
  • 6 kudos

Hi @AJ DJ​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 6 kudos
1 More Replies
smart5mk
by New Contributor III
  • 2268 Views
  • 3 replies
  • 13 kudos

Any Upcoming Updates to Data Engineer Professional Exam or Advanced Data Engineering Course?

Just failed the Data Engineer Professional exam... the Advanced Data Engineering prep course is totally inadequate, and the exam went way beyond in scope with some topics and questions. Are there any changes coming to the curriculum or the exam?

  • 2268 Views
  • 3 replies
  • 13 kudos
Latest Reply
Anonymous
Not applicable
  • 13 kudos

For me, The exam actually really really hard, you must know almost all knowledgeof Databricks Lake House Platform and very deeply. Anyway, mistake make perfect keep fighting. Hope to hear from your post next time with Passed message!

  • 13 kudos
2 More Replies
alonisser
by Contributor II
  • 1444 Views
  • 1 replies
  • 6 kudos

ML inference server (with REST api) how can I specify a GPU cluster.?

is there an API for that? as I couldn't find a way to do this through the UIClassic serving for now (didn't get access to the new "serverless" offering)

  • 1444 Views
  • 1 replies
  • 6 kudos
Latest Reply
alonisser
Contributor II
  • 6 kudos

any clues?

  • 6 kudos
shamly
by New Contributor III
  • 3969 Views
  • 2 replies
  • 3 kudos

spark exception error while reading a parquet file

when I try to read parquet file from Azure datalake container from databricks, I am getting spark exception. Below is my queryimport pyarrow.parquet as pqfrom pyspark.sql.functions import *from datetime import datetimedata = spark.read.parquet(f"/mnt...

  • 3969 Views
  • 2 replies
  • 3 kudos
Latest Reply
DavideAnghileri
Contributor
  • 3 kudos

Hi @shamly pt​ , more info are needed to solve the issue. However common problems are:The storage is not mountThat file doesn't exists in the mounted storageAlso, there is no need to use an f-string if there are no curly brackets with expressions in ...

  • 3 kudos
1 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels