xiangzhu
Contributor III

traditional PAT may have long lifespn, but the new SP feature uses an AAD token which should have a much shorter lifespqn, maybe around one hour, this could be a limiting factor.

However, I haven't tested this yet, so these are merely hypotheses.

Nevertheless, even if it does work with SP, running dbt or notebooks through Git or workspace does not seem like a good idea in terms of operational risk. Packaging everything into an artifact, such as a spark_python_task, appears to be a much better approach.