cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Engineering & Streaming

Forum Posts

Start a conversation
christian_acuna
by • New Contributor
  • 1742 Views
  • 3 replies
  • 0 kudos

Can you z-order a table with Spark Structured Streaming?

Hi, can you do zordering as part of a structured streaming ingestion? 

Data Engineering
Reply
  • 1742 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
Brian2
New Contributor III
  • 0 kudos

You can also turn on auto optimize writes for your table or session.  This is especially good if your table is partitioned.  You should also turn on auto compaction to help control the size of your files.Auto compaction can be enabled at the table or...

  • 0 kudos
2 More Replies
Foundation2023
by • New Contributor II
  • 742 Views
  • 1 replies
  • 1 kudos

Resolved! Diy spark enthusiasts?

On prem Spark enthusiasts? Setting up your own Hadooop is fun, right?

Data Engineering
Reply
  • 742 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
williamong
New Contributor III
  • 1 kudos

This gives me ptsd. Hadoop and mapreduce has been nice and painful!

  • 1 kudos
Pouya
by • New Contributor III
  • 1176 Views
  • 1 replies
  • 3 kudos

Resolved! Is low code/ no code right for you?

There are lots of tools around improving ETL at DAIS.Why is a low code right (or wrong) for you?

Data Engineering
Reply
  • 1176 Views
  • 1 replies
  • 3 kudos
View Replies
Latest Reply
srahgozar
New Contributor III
  • 3 kudos

Hi! I’m a pediatric dentist without any coding experience. Any help I can get with a no-code/low-code solution would be very right for me. Thank you!

  • 3 kudos
RaviM
by • New Contributor
  • 322 Views
  • 0 replies
  • 0 kudos

2023 Keynote

Great stuff at the summit! Like the Satya Nadella interview!!

Data Engineering
Reply
  • 322 Views
  • 0 replies
  • 0 kudos
ayushsood2
by • New Contributor II
  • 710 Views
  • 1 replies
  • 1 kudos

Advanced data engineering training

Completed 2 certifications in 2 days. Thanks to in person training and Databricks academy.

Data Engineering
Reply
  • 710 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
Vbp
New Contributor II
  • 1 kudos

How's the advanced exam? Was it what was covered in the in person training

  • 1 kudos
rgirishram
by • New Contributor
  • 1332 Views
  • 3 replies
  • 0 kudos

Best join approaches

What are some of the best join approaches that we need to be aware of? 

Data Engineering
Reply
  • 1332 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
DeanDing
New Contributor II
  • 0 kudos

This fully depends on the table size and business logic, every case are different. For example, you don’t want to do left join when right table is way smaller than left one. 

  • 0 kudos
2 More Replies
peterlandis
by • New Contributor II
  • 453 Views
  • 0 replies
  • 0 kudos

DLT templating with CDC

DLT offers apply changes.  Leveraging Python you can parameterize a DLT pipeline so that you can create a DLT template to be used for a central pipeline.  My question is around CDC, with the capability to do DLT templating, how can you obtain the pri...

Data Engineering
Reply
  • 453 Views
  • 0 replies
  • 0 kudos
PK2023
by • New Contributor
  • 334 Views
  • 0 replies
  • 0 kudos

I'm at Data+AI Summit 2023 and....

it's awesome. The Keynotes were great and I got to learn about Data Bricks and how it can help my organization. Looking forward to learning more about the possibilities of Data on Cloud and AI

Data Engineering
Reply
  • 334 Views
  • 0 replies
  • 0 kudos
Skystation
by • New Contributor II
  • 548 Views
  • 1 replies
  • 0 kudos

Newbie checkin

Checkin at data ai summit

Data Engineering
Reply
  • 548 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Skystation
New Contributor II
  • 0 kudos

Learned about the experience on how to setup lake house for advertisement data from instacart

  • 0 kudos
Siddhuth
by • New Contributor III
  • 1053 Views
  • 1 replies
  • 3 kudos

Resolved! Delta Clean Room Preview

Which platform will this be available on for preview?

Data Engineering
Reply
  • 1053 Views
  • 1 replies
  • 3 kudos
View Replies
Latest Reply
maxnumbers
New Contributor III
  • 3 kudos

It's available in AWS for public preview, but Azure deployment date is still TBD

  • 3 kudos

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Labels