cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
Anonymous
by • Not applicable
  • 703 Views
  • 0 replies
  • 0 kudos

Daa & AI Summit

Sessions are very useful, welcome reception is very enjoyable. Thanks for the organization.

Data Engineering
Reply
  • 703 Views
  • 0 replies
  • 0 kudos
Phkansakar
by • New Contributor
  • 8335 Views
  • 0 replies
  • 0 kudos

What I learnt

I learned about how to implement and utilize databricks more efficiently. We also explored how to live tables and how to build pipelines in the vastness that we can use this in our data orchestration process.  We also explored autoloading go to live ...

Data Engineering
Reply
  • 8335 Views
  • 0 replies
  • 0 kudos
daniel830
by • New Contributor
  • 1197 Views
  • 0 replies
  • 0 kudos

Learning

I’ve learned so much I can’t put it into words!  Looking forward to more!hopefully more information on data ingestion!

Data Engineering
Reply
  • 1197 Views
  • 0 replies
  • 0 kudos
rgnonohay
by • New Contributor II
  • 765 Views
  • 0 replies
  • 0 kudos

Unity catalog

I need to setup a UC to share data among 4 different accounts. How can I do that using corporate VPC?

Data Engineering
Reply
  • 765 Views
  • 0 replies
  • 0 kudos
Mlaricobar
by • New Contributor
  • 852 Views
  • 0 replies
  • 0 kudos

Spark Optimization strategies

How can we get the correct size for job cluster in a job workflow pipeline based in the complexity of the process? #databricks #deltalake

Data Engineering
Reply
  • 852 Views
  • 0 replies
  • 0 kudos
Maciej
by • New Contributor II
  • 805 Views
  • 0 replies
  • 0 kudos

CDF background implementation

How Delta Lake CDF works? I seen it add additional column to data, where data was updated or deleted. So what is the purpose of change log?

Data Engineering
Reply
  • 805 Views
  • 0 replies
  • 0 kudos
tingtingchan
by • New Contributor
  • 806 Views
  • 0 replies
  • 0 kudos

Greetings for My 1st Data+AI Summit 2023!

Kudos to the amazing instructors and TAs for my first in-person Data Engineer Associate Training, and I've passed my exam! Having a fantastic time so far, can't wait for the content unfolded in the next two days!

Data Engineering
Reply
  • 806 Views
  • 0 replies
  • 0 kudos
Orianh
by • Valued Contributor II
  • 10654 Views
  • 4 replies
  • 3 kudos

function does not exist in JVM ERROR

Hello guys, I'm building a python package that return 1 row from DF at a time inside data bricks environment.To improve the performance of this package i used multiprocessing library in python, I have background process that his whole purpose is to p...

function dont exist in JVM error.
Data Engineering
Reply
  • 10654 Views
  • 4 replies
  • 3 kudos
View Replies
Latest Reply
dineshreddy
New Contributor III
  • 3 kudos

Using thread instead of processes solved the issue for me

  • 3 kudos
3 More Replies
Anonymous
by • Not applicable
  • 3214 Views
  • 2 replies
  • 1 kudos

Delta Tables copying

Hello, I’m trying to copy a table with all it’s versions to unity catalog, I know I can use deep cloning but I want the table with the full history, is that possible?

Data Engineering
Summit23
Reply
  • 3214 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
bikash84
New Contributor III
  • 1 kudos

To copy history, you would have to copy files along with the delta log folder and then create a delta table on that location

  • 1 kudos
1 More Replies
Labels