cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Engineering & Streaming

Forum Posts

carlosancassani
by New Contributor III
  • 1768 Views
  • 3 replies
  • 5 kudos

Error: Credential size is more than configured size limit. As a result credential passthrough won't work for this notebook run.

I get this error when trying to execute parallel slave notebook from a Pyspark "master notebook".note 1: I use same class, functions, cluster, credential for another use case of parallel notebook in the same databricks instance and it works fine.note...

image
  • 1768 Views
  • 3 replies
  • 5 kudos
Latest Reply
Anonymous
Not applicable
  • 5 kudos

Hi @carlosancassani​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...

  • 5 kudos
2 More Replies
Mohit_Jain19
by New Contributor III
  • 2405 Views
  • 8 replies
  • 3 kudos

I passed the Databricks Machine Learning Associate Exam on 2022-11-18. I have not yet received the certificate/Badge. please help Thanks,Mohit Jain

I passed the Databricks Machine Learning Associate Exam on 2022-11-18. I have not yet received the certificate/Badge.please helpThanks,Mohit Jain

  • 2405 Views
  • 8 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Mohit Jain​ Hope all is well! Just wanted to check in that did you receive your badge? If yes then please mark it as the best answer. Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 3 kudos
7 More Replies
Kush
by New Contributor III
  • 2704 Views
  • 5 replies
  • 6 kudos

Unable to find the Databricks Lakehouse Fundamentals Badge (Ceritifcate ID: E-E04YDV)

Hi, I completed my Fundamentals of the Databricks Lakehouse Platform Accreditation on 11/6/2022 but I have not received my badge yet. I have received a certificate which looks kind of fake. Can you please let me know by when will I receive my badge?

  • 2704 Views
  • 5 replies
  • 6 kudos
Latest Reply
Anonymous
Not applicable
  • 6 kudos

Hi @Kushagra Sen​ Hope all is fine!Just a friendly follow-up, have you received your badge? If yes, kindly mark the answer as best and if it hasn't been resolved yet then do let us know for further assistance. Thanks and Regards

  • 6 kudos
4 More Replies
baz
by New Contributor II
  • 940 Views
  • 1 replies
  • 1 kudos

access to my account

access to my accountI can't access to my accountI am getting password reset link in my email but The password reset doesn't work.Invalid email address or passwordNote: Emails/usernames are case-sensitive

  • 940 Views
  • 1 replies
  • 1 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 1 kudos

Hi @messaoud Baz​ , Thank you for reaching out, and we’re sorry to hear about this log-in issue! We have this Community Edition login troubleshooting post on Community. Please take a look, and follow the troubleshooting steps.

  • 1 kudos
ellaine
by New Contributor III
  • 5056 Views
  • 14 replies
  • 9 kudos

Google Ad Manager Reporting API Authentication via Databricks

Hello - I was wondering whether anyone has had any experience fetching data through GAM's Reporting API via Databricks. The Reporting API requires installation of the "googleads" library, as well as a googleads.yaml file. I was able to find some docu...

  • 5056 Views
  • 14 replies
  • 9 kudos
Latest Reply
Anonymous
Not applicable
  • 9 kudos

Hi @Ellaine Ho​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 9 kudos
13 More Replies
Anonymous
by Not applicable
  • 1362 Views
  • 2 replies
  • 16 kudos

Resolved! How to access data files for Databricks Workspace directly through Azure Blob Storage

Hi everyone,This is the first time I used Azure to deploy a Databricks (before I quite familiar use AWS for deploying Databricks). I want to view Databricks Workspace file directly from Azure portal but always don't have a permission.Could you give m...

Image
  • 1362 Views
  • 2 replies
  • 16 kudos
Latest Reply
Unforgiven
Valued Contributor III
  • 16 kudos

@Jensen Ackles​ read document as link below :https://docs.databricks.com/external-data/azure-storage.htmli dont know whats steps ur done, hope this can help u in this case

  • 16 kudos
1 More Replies
TariqueAnwer
by New Contributor II
  • 2986 Views
  • 5 replies
  • 3 kudos

Pyspark CSV Incorrect Count

B1123451020-502,"","{""m"": {""difference"": 60}}","","","",2022-02-12T15:40:00.783Z B1456741975-266,"","{""m"": {""difference"": 60}}","","","",2022-02-04T17:03:59.566Z B1789753479-460,"","",",","","",2022-02-18T14:46:57.332Z B1456741977-123,"","{""...

  • 2986 Views
  • 5 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Tarique Anwer​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...

  • 3 kudos
4 More Replies
ravikanthranjit
by New Contributor III
  • 2880 Views
  • 6 replies
  • 14 kudos

Vacuum on external tables that we mount on ADLS

Want to know the best process of removal of files on ADLS after Optimize and Vacuum Dry run is completed

  • 2880 Views
  • 6 replies
  • 14 kudos
Latest Reply
Anonymous
Not applicable
  • 14 kudos

Hi @Ravikanth Narayanabhatla​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear fr...

  • 14 kudos
5 More Replies
Matt101122
by Contributor
  • 2143 Views
  • 4 replies
  • 4 kudos

Resolved! Why are calls to /api/2.1/jobs/create taking 30-60 seconds to complete?

We are having an issue with calls to /api/2.1/jobs/create taking between 30 and 60 seconds to complete. Calls to /api/2.1/jobs/list complete as expected in about 6 seconds so the issue seems to be only with the job create API. This seems to only be h...

  • 2143 Views
  • 4 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @Matthew Dalesio​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...

  • 4 kudos
3 More Replies
ebg
by New Contributor III
  • 3124 Views
  • 3 replies
  • 10 kudos

I have an Azure storage account, and I need to list the containers on that storage account using scala from databricks notebook

I tried to run azure-cli on databricks (i am using Credential passthrough and my account needs MFA ) and do the following code:az storage container list --account-name "account_name" --auth-mode login --query "[].name" --output tsvHowever, it outputs...

  • 3124 Views
  • 3 replies
  • 10 kudos
Latest Reply
Anonymous
Not applicable
  • 10 kudos

Hi @elias bou ghosn​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...

  • 10 kudos
2 More Replies
AJDJ
by New Contributor III
  • 2475 Views
  • 5 replies
  • 10 kudos

Cost as per the Databricks demo

Hi there,I came across this Databricks demo from the below link. https://youtu.be/BqB7YQ1-KKcKindly Fastforward to time 16:30 or 16:45 of the video and watch few mins of the video related to cost. My understanding is the data is in the lake and datab...

  • 2475 Views
  • 5 replies
  • 10 kudos
Latest Reply
Anonymous
Not applicable
  • 10 kudos

Hi @AJ DJ​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 10 kudos
4 More Replies
smart5mk
by New Contributor III
  • 1583 Views
  • 3 replies
  • 13 kudos

Any Upcoming Updates to Data Engineer Professional Exam or Advanced Data Engineering Course?

Just failed the Data Engineer Professional exam... the Advanced Data Engineering prep course is totally inadequate, and the exam went way beyond in scope with some topics and questions. Are there any changes coming to the curriculum or the exam?

  • 1583 Views
  • 3 replies
  • 13 kudos
Latest Reply
Anonymous
Not applicable
  • 13 kudos

For me, The exam actually really really hard, you must know almost all knowledgeof Databricks Lake House Platform and very deeply. Anyway, mistake make perfect keep fighting. Hope to hear from your post next time with Passed message!

  • 13 kudos
2 More Replies
shamly
by New Contributor III
  • 2963 Views
  • 2 replies
  • 3 kudos

spark exception error while reading a parquet file

when I try to read parquet file from Azure datalake container from databricks, I am getting spark exception. Below is my queryimport pyarrow.parquet as pqfrom pyspark.sql.functions import *from datetime import datetimedata = spark.read.parquet(f"/mnt...

  • 2963 Views
  • 2 replies
  • 3 kudos
Latest Reply
DavideAnghileri
Contributor
  • 3 kudos

Hi @shamly pt​ , more info are needed to solve the issue. However common problems are:The storage is not mountThat file doesn't exists in the mounted storageAlso, there is no need to use an f-string if there are no curly brackets with expressions in ...

  • 3 kudos
1 More Replies
db-avengers2rul
by Contributor II
  • 2471 Views
  • 8 replies
  • 18 kudos

Code snippet error from course - Databricks Academy - Delta Lake Rapid Start with Python

Dear Team,While i was doing hands on practice from the course - Delta Lake Rapid Start with Pythonhttps://customer-academy.databricks.com/learn/course/97/delta-lake-rapid-start-with-pythoni have come across false as the output dbutils.fs.rm(health_t...

  • 2471 Views
  • 8 replies
  • 18 kudos
Latest Reply
Anonymous
Not applicable
  • 18 kudos

Could you give more description about your issue (screenshot or something). Hope to help you find the issue?

  • 18 kudos
7 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels