Hi Team,
We are trying to create DLT pipeline. The scenario is this:-
We already have catalog in unity catalog and under that we have schema called test in which we have 17 tables. Now we are trying to create DLT pipeline and copy the data from those 17 table from test schema to another prd schema.
I already create notebook in which we have this statement
CREATE OR REFRESH STREAMING TABLE prd_acct
as select * from STREAM(catalogname_test.acct);
And same SQL command for all 17 tables. Now when i run the DLT pipeline all the 17 tables has been created in prd schema. However, when graph appear it show couple of tables has been failed with this error.
And when i check the prd schema all the tables appears correctly .
I run the DLT job against cluster only.
Has anyone have any idea why this error happened to me OR what will be correct way to run the DLT pipeline.
Any help will be really appreciated.