cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
Merchiv
by • New Contributor III
  • 3205 Views
  • 4 replies
  • 0 kudos

Difference between Databricks and local pyspark split.

I have noticed some inconsistent behavior between calling the 'split' fuction on databricks and on my local installation.Running it in a databricks notebook givesspark.sql("SELECT split('abc', ''), size(split('abc',''))").show()So the string is split...

image.png
Data Engineering
Reply
  • 3205 Views
  • 4 replies
  • 0 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Ivo Merchiers​ :The behavior you are seeing is likely due to differences in the underlying version of Apache Spark between your local installation and Databricks. split() is a function provided by Spark's SQL functions, and different versions of Spa...

  • 0 kudos
3 More Replies
moski
by • New Contributor II
  • 6675 Views
  • 9 replies
  • 7 kudos

Databricks short cut to split a cell

Is there a shortcut to split a cell into two in Dtabricks notebook as in Jupiter notebook? in jupyter notebook it is Shift/Ctr/-

Data Engineering
Reply
  • 6675 Views
  • 9 replies
  • 7 kudos
View Replies
Latest Reply
Harshjot
Contributor III
  • 7 kudos

 Hi @mundy Jim​ / All, Attached are two snapshots so first snapshot with one cell if pressed Ctrl+Alt+Minus split into two.  

  • 7 kudos
8 More Replies
weldermartins
by • Honored Contributor
  • 3175 Views
  • 5 replies
  • 13 kudos

Hello everyone, I have a directory with 40 files. File names are divided into prefixes. I need to rename the prefix k3241 according to the name in the...

Hello everyone, I have a directory with 40 files.File names are divided into prefixes. I need to rename the prefix k3241 according to the name in the last prefix.I even managed to insert the csv extension at the end of the file. but renaming files ba...

Template
Data Engineering
Reply
  • 3175 Views
  • 5 replies
  • 13 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 13 kudos

Hi @welder martins​ How are you doing?Thank you for posting that question. We are glad you could resolve the issue. Would you want to mark an answer as the best solution?Cheers

  • 13 kudos
4 More Replies
Atul_Sharan
by • New Contributor II
  • 4152 Views
  • 5 replies
  • 2 kudos

Resolved! Hello Experts - I am facing one technical issue with Databricks SQL - IF-ELSE or CASE statement implementation when trying to execute two separate set of queries based on a valued of a column of the Delta table.

Hi Experts,I'm reading a pipe delimited source file where first row does not contain the data but contains the - REPLACE or UPDATE values which indicate the refresh type if it is Full refresh Or Upsert. The second row is nothing but header and the a...

Data Engineering
Reply
  • 4152 Views
  • 5 replies
  • 2 kudos
View Replies
Latest Reply
Atul_Sharan
New Contributor II
  • 2 kudos

Thanks a ton Gurpreet, Suggested approach was helpful in resolution of the issue. Appreciate your help !!

  • 2 kudos
4 More Replies
Labels