Hey there,
I am currently using the Databricks API to trigger a specific DBT job. For this, I am calling the API in a Web Activity on Azure datafactory and sending as headers the token and for the body I am sending the Job ID and the necessary vars I need for DBT.
Sometimes I get a "Internal Error", I cannot replicate the error since I don't know the cause, but when I do a retrial it works perfectly.
More details: I was using 2.1 version of the API, I already tried downgrading for 2.0 still get the same error sometimes. Also, I don't believe the problem is related to the body/headers since the retrial works and uses the exact same body/headers.
What could be a possible solution for this issue? Any workarrounds I can use to make sure this error does not occur?
