DBT Jobs || API call returns "Internal Error"

dcc — Mon, 04 Dec 2023 10:23:17 GMT

Hey there,

I am currently using the Databricks API to trigger a specific DBT job. For this, I am calling the API in a Web Activity on Azure datafactory and sending as headers the token and for the body I am sending the Job ID and the necessary vars I need for DBT.

Sometimes I get a "Internal Error", I cannot replicate the error since I don't know the cause, but when I do a retrial it works perfectly.

More details: I was using 2.1 version of the API, I already tried downgrading for 2.0 still get the same error sometimes. Also, I don't believe the problem is related to the body/headers since the retrial works and uses the exact same body/headers.

What could be a possible solution for this issue? Any workarrounds I can use to make sure this error does not occur?

Re: DBT Jobs || API call returns "Internal Error"

jose_gonzalez — Tue, 12 Dec 2023 19:21:21 GMT

Could you please share the driver logs? it will help us to narrow down the issue

topic DBT Jobs || API call returns "Internal Error" in Data Engineering

DBT Jobs || API call returns "Internal Error"

Re: DBT Jobs || API call returns "Internal Error"