Hi, @adisalj. Facing issues with TABLE_OR_VIEW_NOT_FOUND errors after cloning data objects can be frustrating.
Let’s explore some potential reasons and solutions:
Schema Mismatch:
- Ensure that the schema of the cloned tables and views matches the schema expected by your DBT transformations.
- Verify that the non-production catalogue has the same schema structure as the production catalog.
Dependency Order:
- Check if there are dependencies between tables/views within the schema. Sometimes, a table or view might reference another one.
- Ensure that the order of cloning considers these dependencies. Cloning tables/views in the correct order can prevent errors.
Permissions and Access:
- Confirm that the DBT transformations user has the necessary permissions to access the cloned tables/views.
- Verify that the non-production catalog has the same access rights as the production catalog.
Metadata Refresh:
- After cloning, refresh the metadata in your non-production catalog. This ensures that the catalog is aware of the newly cloned objects.
- Some platforms might require manual metadata refresh or cache clearing.
Logging and Debugging:
- Enable detailed logging during the cloning process. Check the logs for any specific error messages related to TABLE_OR_VIEW_NOT_FOUND.
- Investigate the logs to identify which specific table or view is causing the issue.
Automated Validation:
- Consider setting up automated validation checks after cloning. For example, run a script or query that verifies the existence and structure of the cloned objects.
- If any discrepancies are found, trigger an alert or notification.
Database Engine Specifics:
- Different database engines (e.g., PostgreSQL, MySQL, SQL Server) might have specific behaviour during cloning.
- Research engine-specific documentation or community forums to see if there are known cloning-related issues.
Version Control and Rollbacks:
- Maintain version control for your catalog objects. If issues arise, you can roll back to a known working state.
- Regularly test the cloning process in a non-production environment to catch any issues early.
Remember that debugging such issues often involves a combination of trial and error, thorough investigation, and collaboration with your database administrators or platform support. If the problem persists, contact the Databricks support team for further assistance. 🛠️🔍