03-16-2023 10:47 PM
I have tried 2 tables with different catalogs, schemas, column names and it throws LOCATION_OVERLAP. In spark we can create as many tables as needed on the same location.
03-17-2023 08:32 AM
@Ashish Singh :
In Databricks, it is not recommended to create multiple tables on the same location in the same database and schema. The Unified Analytics Warehouse (UAW) used by Databricks enforces a one-to-one mapping between tables and directories in the underlying file system. If you try to create multiple tables with the same location in the same database and schema, you will receive a "LOCATION_OVERLAP" error message because it conflicts with this one-to-one mapping.
However, it is possible to create multiple tables on the same location in different databases or schemas in Databricks. In this case, each table is treated as a separate object in the UAW and has its own unique metadata associated with it.
If you need to store multiple tables with the same data in Databricks, you may consider creating views on top of the original table, rather than creating additional tables on the same location. Views can provide an efficient and flexible way to manipulate data without duplicating the underlying data.
03-17-2023 10:41 AM
Thanks for answering. But as you mentioned to create multiple tables in different database/schema - This does not seem to work either. I am not able to create tables in different catalogs with same location, same for schema.
03-17-2023 11:16 PM
Hi @Ashish Singh
Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help.
We'd love to hear from you.
Thanks!
03-17-2023 11:22 PM
Hi,
I still am not sure if we can or cannot create tables on the same location when using different catalogs or schemas in unity catalog. Suteja mentioned we can but I don't seem to be able to, it still gives the location overlap error. This seems to be specific to unity catalog too, as hive metastore allows it.
If you can please clarify this?
Thanks
03-19-2023 08:46 PM
Hi @Ashish Singh
I'm sorry you could not find a solution to your problem in the answers provided.
Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.
I suggest providing more information about your problem, such as specific error messages, error logs or details about the steps you have taken. This can help our community members better understand the issue and provide more targeted solutions.
Alternatively, you can consider contacting the support team for your product or service. They may be able to provide additional assistance or escalate the issue to the appropriate section for further investigation.
Thank you for your patience and understanding, and please let us know if there is anything else we can do to assist you.
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group