cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

joselita
by New Contributor III
  • 19498 Views
  • 6 replies
  • 10 kudos

AnalysisException: is not a Delta table.

Hello, I changed the DBR from 7.2 to 10.4 and I receive the following error: AnalysisException: is not a Delta table. The table is create , using DELTA. so for sure is a Delta table, even though, I read that I read that from vers. 8 all tables are De...

STG_DATA_LOAD
  • 19498 Views
  • 6 replies
  • 10 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 10 kudos

Hi @JOSELITA MOLTISANTI​, I wanted to share this S.O thread with you which is similar to the issue mentioned here by you. Please have a look and revert back if you were able to resolve your issue.

  • 10 kudos
5 More Replies
deisou
by New Contributor
  • 1262 Views
  • 1 replies
  • 0 kudos

How to change Databricks Academy username(Email)?

Is there a way to change my Databricks Academy username(email)? It is greyed out in my profile and I cannot update it. How do I go about getting it updated?

  • 1262 Views
  • 1 replies
  • 0 kudos
Latest Reply
Debayan
Esteemed Contributor III
  • 0 kudos

Hi, Please go through the Databricks Academy FAQs here: https://files.training.databricks.com/lms/docebo/databricks-academy-faq.pdfAlso, please go through the post here: https://community.databricks.com/s/feed/0D53f00001dq6W6CAI.

  • 0 kudos
LearningDatabri
by Contributor II
  • 1540 Views
  • 2 replies
  • 1 kudos

Resolved! why this change in UI

The change in the UI is really confusing on what to use where. Earlier i had HC clusters and now I cant find it in new UI. It says HC clusters are not available. I want to use the HC cluster functions. where can I get that?

  • 1540 Views
  • 2 replies
  • 1 kudos
Latest Reply
Prabakar
Esteemed Contributor III
  • 1 kudos

Hi @Databricks learner​ see if this post helps you.

  • 1 kudos
1 More Replies
yatharthmahesh
by New Contributor III
  • 1908 Views
  • 4 replies
  • 6 kudos

ENABLE CHANGE DATA FEED FOR EXISTING DELTA-TABLE

I have a delta table already created, now I want to enable the change data feed. I read that I have to set delta.enableChangeDataFeed property to true. But however, this cannot be done using the Scala API. I tried using this but it didn't work. I am ...

  • 1908 Views
  • 4 replies
  • 6 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 6 kudos

Hi @Yatharth Maheshwari​, We haven’t heard from you on the last response from @Jose Gonzalez​, and I was checking back to see if his suggestions helped you. Or else, If you have any solution, please do share that with the community as it can be helpf...

  • 6 kudos
3 More Replies
_Orc
by New Contributor
  • 13722 Views
  • 6 replies
  • 3 kudos

Resolved! Precision and scale is getting changed in the dataframe while casting to decimal

When i run the below query in databricks sql the Precision and scale of the decimal column is getting changed.Select typeof(COALESCE(Cast(3.45 as decimal(15,6)),0));o/p: decimal(16,6)expected o/p: decimal(15,6)Any reason why the Precision and scale i...

  • 13722 Views
  • 6 replies
  • 3 kudos
Latest Reply
berserkersap
Contributor
  • 3 kudos

You can use typeof(COALESCE(Cast(3.45 as decimal(15,6)),0.0)); (instead of 0)

  • 3 kudos
5 More Replies
niels
by New Contributor III
  • 2715 Views
  • 5 replies
  • 12 kudos

Resolved! Change cluster mid-pipeline

I have a notebook functioning as a pipeline, where multiple notebooks are chained together. The issue I'm facing is that some of the notebooks are spark-optimized, others aren't, and what I want is to use 1 cluster for the former and another for the ...

  • 2715 Views
  • 5 replies
  • 12 kudos
Latest Reply
Kaniz_Fatma
Community Manager
  • 12 kudos

Hi @Niels Ota​ , We haven’t heard from you on the last response from @Prabakar Ammeappin​ , and I was checking back to see if you have a resolution yet. If you have any solution, please share it with the community as it can be helpful to others. Othe...

  • 12 kudos
4 More Replies
petilodie
by New Contributor III
  • 1181 Views
  • 1 replies
  • 3 kudos

Resolved! How can I change the admin settings by Terraform?

I need to update most of the settings that are visible on the Admin Console UI by using Terraform. In another post in this forum I saw that I can use `custom_config` in a `databricks_workspace_conf` resource to achieve that but the options seem limit...

  • 1181 Views
  • 1 replies
  • 3 kudos
Latest Reply
petilodie
New Contributor III
  • 3 kudos

Ok, looks like I can inspect the network and see which flags are sent to the endpoint. Tried that and it worked.

  • 3 kudos
Hola1801
by New Contributor
  • 1373 Views
  • 3 replies
  • 3 kudos

Resolved! Float Value change when Load with spark? Full Path?

Hello,I have created my table in Databricks, at this point everything is perfect i got the same value than in my CSV. for my column "Exposure" I have :0 0,00 1 0,00 2 0,00 3 0,00 4 0,00 ...But when I load my fi...

  • 1373 Views
  • 3 replies
  • 3 kudos
Latest Reply
jose_gonzalez
Moderator
  • 3 kudos

Hi @Anis Ben Salem​ ,How do you read your CSV file? do you use Pandas or Pyspark APIs? also, how do you created your table?could you share more details on the code you are trying to run?

  • 3 kudos
2 More Replies
MarcLimotte
by New Contributor II
  • 21269 Views
  • 12 replies
  • 0 kudos

Why do I get 'java.io.IOException: File already exists' for saveAsTable with Overwrite mode?

I have a fairly small, simple DataFrame, month:month.schema org.apache.spark.sql.types.StructType = StructType(StructField(month,DateType,true), StructField(real_month,TimestampType,true), StructField(month_millis,LongType,true))The month Dataframe i...

  • 21269 Views
  • 12 replies
  • 0 kudos
Latest Reply
ReKa
New Contributor III
  • 0 kudos

Your schema is tight, but make sure that the conversion to it does not throw an exception. Try with Memory Optimized Nodes, you may be fine. My problem was parsing a lot of data from sequence files containing 10K xml files and saving them as a table...

  • 0 kudos
11 More Replies
Labels