cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Mapajr
by New Contributor III
  • 3175 Views
  • 2 replies
  • 3 kudos

Issues pushing repos on Gitlab with Databricks

Our company uses Gitlab enterprise edition and we link our repos up to databricks through this. Randomly we will get errors when trying to push the repo and we have to spend hours debugging trying to figure out what is causing the push error on datab...

  • 3175 Views
  • 2 replies
  • 3 kudos
Latest Reply
Vidula
Honored Contributor
  • 3 kudos

Hey there @Mark Patrick​ Hope you are well. Just wanted to see if you were able to find an answer to your question and would you like to mark an answer as best? It would be really helpful for the other members too.Cheers!

  • 3 kudos
1 More Replies
ViktorWolf
by New Contributor
  • 1742 Views
  • 3 replies
  • 0 kudos

Why sometimes autoloader lose the checkpoint path and break the streaming?

Why sometimes autoloader lose the checkpoint path and break the streaming?

  • 1742 Views
  • 3 replies
  • 0 kudos
Latest Reply
Vidula
Honored Contributor
  • 0 kudos

Hi @Vittorio Antonacci​ Hope all is well! Just wanted to check in if you were able to resolve your issue, and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from y...

  • 0 kudos
2 More Replies
al_joe
by Contributor
  • 1007 Views
  • 0 replies
  • 3 kudos

Why is this simple numerical operation not precise?

I was experimenting with beginner tutorial and saw this strange output ...Why is this so ? And why is the behavior not consistent for ALL rows updated by the same statement?8.8 - 1 = 7.800000000000001See screenshot ...

20220827_180902_msedge_DE_2.1_-_Managing_Delta_Tables_-_Databricks_-_Pers
  • 1007 Views
  • 0 replies
  • 3 kudos
aayrm5
by Honored Contributor
  • 6452 Views
  • 6 replies
  • 4 kudos

Issue with .dbc in the Advanced Data Engineering course in Databricks Academy

The very first notebook of the dbc notebook which is a setup cell fails.

image
  • 6452 Views
  • 6 replies
  • 4 kudos
Latest Reply
Niha1
New Contributor III
  • 4 kudos

Hi Riyaz,Please find the snippet of the error below--:"AnalysisException: Path does not exist: dbfs:/user/nniha9188@gmail.com/dbacademy/machine_learning/datasets/airbnb/sf-listings/sf-listings-2019-03-06-clean.parquet"Source-The source for this datas...

  • 4 kudos
5 More Replies
Shakzz
by New Contributor III
  • 5200 Views
  • 2 replies
  • 10 kudos
  • 5200 Views
  • 2 replies
  • 10 kudos
Latest Reply
Vidula
Honored Contributor
  • 10 kudos

Hey there @Shakti Chand​ Hope all is well! Just wanted to check in if you were able to resolve your issue, and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from y...

  • 10 kudos
1 More Replies
ankit_k
by New Contributor
  • 1805 Views
  • 3 replies
  • 0 kudos

Move a GCP Project with Databricks in it to new Organization

We are trying to move a GCP project to a Newly created Org and new billing account. We have a Databricks instance from GCP Marketplace with licensing As per the docs when we change a billing account for a Project the license on the first billing acco...

  • 1805 Views
  • 3 replies
  • 0 kudos
Latest Reply
Vidula
Honored Contributor
  • 0 kudos

Hi @Ankit K​ Hope you are well. Just wanted to see if you were able to find an answer to your question and would you like to mark an answer as best?  Else please let us know if you need more help. We'd love to hear from you.Cheers!

  • 0 kudos
2 More Replies
alan_cooper
by New Contributor II
  • 2467 Views
  • 2 replies
  • 2 kudos

Can you manage Workspaces using Terraform on an AWS Standard Tier account.

Background: I am looking to migrate a small PoC Databricks workspace in Azure over to a productionised deployment in AWS.I need to deploy all the Workspaces and all the notebooks etc code in those workspaces using a CI/CD pipeline, using Terraform to...

  • 2467 Views
  • 2 replies
  • 2 kudos
Latest Reply
Vidula
Honored Contributor
  • 2 kudos

Hi @ALAN COOPER​ I hope all is well! I just wanted to check in if you were able to resolve your issue would you be happy to mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 2 kudos
1 More Replies
tafmuko
by New Contributor II
  • 1047 Views
  • 0 replies
  • 2 kudos

Azure Databricks Tenant/Directory Swicth

HelloWe are looking to move a PAYG subscription to a new tenant. We've got Azure Databricks deployed. There are a few questions:Will Databricks continue to function properly (aside from the RBAC list which we will re-apply to all resources)Since Data...

  • 1047 Views
  • 0 replies
  • 2 kudos
ABose8
by Contributor
  • 7007 Views
  • 12 replies
  • 6 kudos

Resolved! Attended 27th July 2022 webinar but have not recieved voucher,even uploaded Lakehouse certificate

@Kaniz Fatma​ @Samantha Menot​ This is Arindam Bose.Actually I attended Databricks webinar on 27th July for (Databricks Certification Exam Overview Training: Databricks Certified Data Analyst Associate).I was expecting vouchers for Databricks Certifi...

  • 7007 Views
  • 12 replies
  • 6 kudos
Latest Reply
st1
New Contributor III
  • 6 kudos

Hello Team, I am also facing the same issue.  I met all the needs to qualify for a certificate voucher but have not received yet - Attended 3 days webinar Data Analysis With Databricks SQL Training & Certification Overview as part of Databricks Eleva...

  • 6 kudos
11 More Replies
isaac_gritz
by Databricks Employee
  • 2135 Views
  • 1 replies
  • 6 kudos

Versions of Spark, Python, Scala, R in each Databricks Runtime

What version of Spark, Python, Scala, R are included in each Databricks Runtime? What libraries are pre-installed?You can find this info at the Databricks runtime releases page (AWS | Azure | GCP).Let us know if you have any additional questions on t...

  • 2135 Views
  • 1 replies
  • 6 kudos
Latest Reply
maxdata
Databricks Employee
  • 6 kudos

Wow! Thanks for the help @Isaac Gritz​ !

  • 6 kudos
nhgil
by New Contributor II
  • 2836 Views
  • 3 replies
  • 0 kudos

Accessing Workspace?

Trying to follow along with the instructions for the 'AWS and Databricks' training yet the documentation gives no indication of how to access the workspace that is ostensibly generated in AWS. I subscribed through the marketplace and cloud formation...

  • 2836 Views
  • 3 replies
  • 0 kudos
Latest Reply
nhgil
New Contributor II
  • 0 kudos

Unfortunately by the time I received a response my trial had long lapsed so I had already unsubscribed in aws.

  • 0 kudos
2 More Replies
Chandana
by New Contributor II
  • 1194 Views
  • 1 replies
  • 3 kudos

What’s the USP? DB SQL serverless? When is it coming to Azure?

What’s the USP? DB SQL serverless? When is it coming to Azure?

  • 1194 Views
  • 1 replies
  • 3 kudos
Latest Reply
User16752242622
Valued Contributor
  • 3 kudos

Hi @Chandana Basani​ DB SQL Serverless in Azure is planned for GA on the Release quarter FY23 Q1 in the release month 2023-01Best,Akash

  • 3 kudos
sawya
by New Contributor II
  • 3683 Views
  • 3 replies
  • 0 kudos

Migrate workspaces to another AWS account

Hi everyone,I have a Databricks workspace in an AWS account that I have to migrate to a new AWS accountDo you know how I can do it ? Or it's better to recreate a new one and move all the workbooks and if I choose to create one new how can you export ...

  • 3683 Views
  • 3 replies
  • 0 kudos
Latest Reply
Abishek
Databricks Employee
  • 0 kudos

@AMADOU THIOUNE​ Can you check the below link to export the run jobs? https://docs.databricks.com/jobs.html#export-job-runs. Try to reuse the same job_id with the /update and /reset endpoints, it should allow you much better access to previous run re...

  • 0 kudos
2 More Replies
rdobbss
by New Contributor II
  • 1896 Views
  • 2 replies
  • 0 kudos

RPC Disassociate error due to container threshold exceeding and garbage collector error when reading 23 gb multiline JSON file.

I am reading 23 gb multi line json file and flattening it using udf and writing datframe as parquet using psypark.Cluster I am using is 3 node (8 core) 64gb memory with limit to go upto 8 nodes.I am able to process 7gb file with no issue and takes ar...

  • 1896 Views
  • 2 replies
  • 0 kudos
Latest Reply
Vidula
Honored Contributor
  • 0 kudos

Hi @Ravi Dobariya​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...

  • 0 kudos
1 More Replies
Azeez
by New Contributor II
  • 7935 Views
  • 8 replies
  • 1 kudos

Resolved! BAD_REQUEST:Failed to get oauth access token.Please try logout and login again

We deployed a test databricks workspace cluster on GCP. A single cluster got spinned up.Later we deleted the workspace.Now when we are trying to create a new one.It is giving this error"BAD_REQUEST:Failed to get oauth access token.Please try logout ...

  • 7935 Views
  • 8 replies
  • 1 kudos
Latest Reply
Prabakar
Databricks Employee
  • 1 kudos

@Azeez Sayyad​ you can try this workaround.Remove the Databricks App from your Google account. In Google account settings, go to "Manage third-paarty access", and remove Databricks from both Third-Paarty app with account access and Sign-in with Googl...

  • 1 kudos
7 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels