topic Re: Why companies use databricks SQL and Redshift at the same time? in Warehousing & Analytics

Why companies use databricks SQL and Redshift at the same time?

eimis_pacheco — Sat, 15 Jul 2023 21:24:02 GMT

Hi community,

I suddenly found myself confused, and this might sound like an obvious answer for some but not for me at least in this moment.

I am not getting why companies use databricks SQL and Redshift at the same time?

I mean, with databricks platform once you have ran your spark job a delta table is created (as the default databricks format) for ACID support, then these tables can be used and query in databricks SQL by data analyst and data scientists. They can create dashboards directly in there. They can even connect to it with BI tools like tableau, ect...

It looks like a data warehouse to me already, since data is already in a structure format ready and available for the usage of data analyst and data scientists.

Then, why companies use a big data plaform like databricks and use a data warehouse like Redshift u other at the same time? can someone clarify this to me, please?

Thank you very much in advance

Regards

Re: Why companies use databricks SQL and Redshift at the same time?

-werners- — Mon, 17 Jul 2023 07:27:23 GMT

I think in most of the cases it is simple: Redshift exists for years, whereas databricks sql is pretty new.
So a lot of companies were already using redshift when dbrx sql popped up.
Once you have something in production, it is not an easy task to phase it out. So when you decide to start using databricks sql, the redshift will still exist for a while.
F.e. we also have a Synapse database running while using db sql. We want to get rid of it, but that takes some time because we have to migrate/adapt all queries running on that synapse

There might also be technical features that are possible and used in redshift, which are not (yet) available in db sql.

Re: Why companies use databricks SQL and Redshift at the same time?

Anonymous — Tue, 18 Jul 2023 05:13:32 GMT

Hi @eimis_pacheco

We haven't heard from you since the last response from @-werners- , and I was checking back to see if her suggestions helped you.

Or else, If you have any solution, please share it with the community, as it can be helpful to others.

Also, Please don't forget to click on the "Select As Best" button whenever the information provided helps resolve your question.

Re: Why companies use databricks SQL and Redshift at the same time?

eimis_pacheco — Tue, 18 Jul 2023 20:44:34 GMT

Thank you very much for your reply, It helps. Best Regards