topic Re: What exactly is Vectorized query processing and columnar acceleration in Get Started Discussions

What exactly is Vectorized query processing and columnar acceleration

dvl_priyansh — Tue, 13 Aug 2024 06:20:14 GMT

Hey folks! I want to know and understand while using photon acceleration, there is a feature called columnar acceleration which basically is a method of storing data in columns rather than rows, which is particularly advantageous for analytical databases and data warehouses. I want to know that how it actually works and what makes it efficient during the ETL process?

Where as Vectorized query processing operates on batches of rows (vectors) rather than processing each row individually but how it actually makes a difference, whether we read the data in batches of by rows individually, anyhow we are reading each row only, so what makes there two features efficient in the case of photon acceleration?

Re: What exactly is Vectorized query processing and columnar acceleration

szymon_dybczak — Tue, 13 Aug 2024 06:48:11 GMT

Hi @dvl_priyansh ,

Take a look at below article. It has a great explanation and answers you questions:

A Closer Look Into Databricks’s Photon Engine | by Vu Trinh | Data Engineer Things (det.life)

Re: What exactly is Vectorized query processing and columnar acceleration

Retired_mod — Wed, 14 Aug 2024 08:13:49 GMT

Hi @szymon_dybczak, Thanks for reaching out! Please review the response and let us know if it answers your question. Your feedback is valuable to us and the community.

If the response resolves your issue, kindly mark it as the accepted solution. This will help close the thread and assist others with similar queries.

We appreciate your participation and are here if you need further assistance!

Re: What exactly is Vectorized query processing and columnar acceleration

szymon_dybczak — Wed, 14 Aug 2024 14:57:33 GMT

Hi @Retired_mod ,

By mistake you've mentioned my name, but it was @dvl_priyansh who actually asked a question 🙂

I've only provided an answer 😉