cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
vijaykumarbotla
by • New Contributor III
  • 2194 Views
  • 5 replies
  • 1 kudos

Resolved! Getting error : Analysis Exception : olumn Is There a PO#17748 are ambiguous. It's probably because you joined several Datasets together, and some of these Datasets are the same. This column points to one of the Datasets but Spark.

AnalysisException: Column Is There a PO#17748 are ambiguous. It's probably because you joined several Datasets together, and some of these Datasets are the same. This column points to one of the Datasets but Spark is unable to figure out which one. ...

Data Engineering
Reply
  • 2194 Views
  • 5 replies
  • 1 kudos
View Replies
Latest Reply
vijaykumarbotla
New Contributor III
  • 1 kudos

Hi All,the solution for this problem is very strange.this has caused due to the version of the Databricks runtime.We are using Runtime version 7.0 with Apache Spark 3.0.0 version.In PRD we are using Runtime version 11.3LTS with Apache Spark 3.3.0 ver...

  • 1 kudos
4 More Replies
StephanieRivera
by • Valued Contributor II
  • 2868 Views
  • 4 replies
  • 2 kudos

Resolved! How do I download and unzip datasets from Kaggle into DBFS?

Data Engineering
Reply
  • 2868 Views
  • 4 replies
  • 2 kudos
View Replies
Latest Reply
Debayan
Esteemed Contributor III
  • 2 kudos

Hi, You can refer to https://docs.databricks.com/files/unzip-files.html. You can curl the file you want and then it can be unzipped as mentioned in the doc. Please let us know if this helps.Also, please tag @Debayan with your next update which will n...

  • 2 kudos
3 More Replies
quakenbush
by • Contributor
  • 2083 Views
  • 4 replies
  • 5 kudos

Resolved! Does Databricks offer something like Oracle's dblink?

I am aware, I can load anything into a DataFrame using JDBC, that works well from Oracle sources. Is there an equivalent in Spark SQL, so I can combine datasets as well?Basically something like so - you get the idea...select lt.field1, rt.fie...

Data Engineering
Reply
  • 2083 Views
  • 4 replies
  • 5 kudos
View Replies
Latest Reply
Kaniz
Community Manager
  • 5 kudos

Hi @Roger Bieri​  (Customer)​, I appreciate your attempt to choose the best answer for us. I'm glad you got your query resolved. @Joseph Kambourakis​ and @Adrian Łobacz​, Thank you for giving excellent answers .

  • 5 kudos
3 More Replies
Kavin
by • New Contributor II
  • 957 Views
  • 2 replies
  • 2 kudos

Issue converting the datasets into JSON

Im a newbie to Databricks, I need to convert the data sets into JSON. i tried bth FOR JSON AUTO AND FOR JSON PATH, However im getting an issue - [PARSE_SYNTAX_ERROR] Syntax error at or near 'json'line My Query works fine without FOR JSON AUTO AND FOR...

Data Engineering
Reply
  • 957 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
Kaniz
Community Manager
  • 2 kudos

Hi @Kavin Natarajan​, We haven’t heard from you since the last response from @Debayan Mukherjee​​, and I was checking back to see if their suggestions helped you. Or else, If you have any solution, please share it with the community, as it can be hel...

  • 2 kudos
1 More Replies
Geeya
by • New Contributor II
  • 957 Views
  • 1 replies
  • 0 kudos

After several iteration of filter and union, the data is bigger than spark.driver.maxResultSize

The process for me to build model is:filter dataset and split into two datasetsfit model based on two datasets union two datasetsrepeat 1-3 stepsThe problem is that after several iterations, the model fitting time becomes longer dramatically, and the...

Data Engineering
Reply
  • 957 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Ryan_Chynoweth
Honored Contributor III
  • 0 kudos

I assume that you are using PySpark to train a model? It sounds like you are collecting data on the driver and likely need to increase the size. Can you share any code?

  • 0 kudos
Labels