We are testing an ingestion from kafka to databricks using a streaming table. The streaming table was created by a DAB deployed to "production" which runs as a service principal. This means the service principal is the "owner" of the table.

We now want to move that streaming table to a different pipeline without losing the data that has been saved in the existing streaming table. I see there is some documentation for that here: Move tables between Lakeflow Declarative Pipelines - Azure Databricks | Microsoft Learn

The problem is that I personally cannot execute the SQL to alter the streaming table (PERMISSION_DENIED: User is not an owner of Pipeline '....'.). Unfortunately it also seems like one cannot change the owner of an existing streaming table ([STREAMING_TABLE_OPERATION_NOT_ALLOWED.UNSUPPORTED_OPERATION] The operation SET OWNER is not allowed: The operation is not supported on Streaming Tables. SQLSTATE: 42601

Is there another way to alter the pipeline id for the existing table? I guess a workaround would be to run the ALTER TABLE script as part of the DAB so that it is executed as the service princpal?