โ02-27-2022 02:40 AM
โ02-27-2022 06:05 AM
Hive metastore is semantic layer, it is like mapping so you have folder with delta or json etc. and it is mapped as table so you can query your data using SQL syntax.
SQL endpoint is is server address for incoming SQL queries (like jdbc/odbc) so you can query tables registered in hive metastore from any application (PowerBI, qlikview, looker, or just from own code using jdbc/odbc driver)
โ02-27-2022 06:05 AM
Hive metastore is semantic layer, it is like mapping so you have folder with delta or json etc. and it is mapped as table so you can query your data using SQL syntax.
SQL endpoint is is server address for incoming SQL queries (like jdbc/odbc) so you can query tables registered in hive metastore from any application (PowerBI, qlikview, looker, or just from own code using jdbc/odbc driver)
โ02-28-2022 04:54 AM
What Hubert said is correct. I'd also add that Apache Hive is an old/obsolete tool created at Facebook and open sourced that was a SQL interface for MapReduce. https://hive.apache.org/
It's often helpful to think of SparkSQL as the modern or evolved version of Hive.
SQL Endpoint is more of a cluster to run SQL queries on https://docs.databricks.com/sql/admin/sql-endpoints.html
โ02-28-2022 05:19 AM
@Joseph Kambourakisโ hi can we not run hive queries with Spark as execution engine ? what difference does it make when compared with data bricks SQL end point ... apart from delta engine .
โ02-28-2022 05:47 AM
Yes, in later updates Hive did get a Spark backend engine, but at that time it was largely obsolete.
SQL endpoints have many advantages. They are cloud optimized and there is a C++ engine (photon) that is faster than the traditional spark engine. The endpoints are more compute environments, where hive was more of a syntax and query engine. Also, Spark SQL became more Ansi compatible in recent releases. https://spark.apache.org/docs/latest/sql-ref-ansi-compliance.html
โ03-08-2022 01:50 PM
@Basavaraj Angadiโ - Does @Joseph Kambourakisโ's response answer your question well enough to mark it as best? If not, how may we help you more?
โ03-29-2022 03:13 AM
Thank you for sharing your valuable solution, it's work properly.
โ03-29-2022 03:50 AM
Great to hear ๐
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.