What exactly is Vectorized query processing and columnar acceleration
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-12-2024 11:20 PM
Hey folks! I want to know and understand while using photon acceleration, there is a feature called columnar acceleration which basically is a method of storing data in columns rather than rows, which is particularly advantageous for analytical databases and data warehouses. I want to know that how it actually works and what makes it efficient during the ETL process?
Where as Vectorized query processing operates on batches of rows (vectors) rather than processing each row individually but how it actually makes a difference, whether we read the data in batches of by rows individually, anyhow we are reading each row only, so what makes there two features efficient in the case of photon acceleration?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-12-2024 11:48 PM
Hi @dvl_priyansh ,
Take a look at below article. It has a great explanation and answers you questions:
A Closer Look Into Databricks’s Photon Engine | by Vu Trinh | Data Engineer Things (det.life)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-14-2024 01:13 AM
Hi @szymon_dybczak, Thanks for reaching out! Please review the response and let us know if it answers your question. Your feedback is valuable to us and the community.
If the response resolves your issue, kindly mark it as the accepted solution. This will help close the thread and assist others with similar queries.
We appreciate your participation and are here if you need further assistance!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
08-14-2024 07:57 AM
Hi @Retired_mod ,
By mistake you've mentioned my name, but it was @dvl_priyansh who actually asked a question 🙂
I've only provided an answer 😉