Create a docker image for dbt task

essura — Wed, 21 Feb 2024 13:14:01 GMT

Hi there,

We are trying to setup up a docker image for our dbt execution, primarily to improve execution speed, but also to simplify deployment (we are using a private repos for both the dbt project and some of the dbt packages).

It seems to work currently, but fx downloading artifacts is not working and there are some complaints in log4j surrounding missing R.

Is it possible to see the built-in dbt task Dockerfile? or at least get an outline of used directories, env variables and dependencies?

/Esben

Re: Create a docker image for dbt task

essura — Thu, 22 Feb 2024 10:50:50 GMT

This looks kind of what ChatGPT proposed as a solution to me.

My issue is that i'm trying to use our custom container in a dbt task (a workflow task). The container is working, but we can't fx use the "download artifact", probably because the path is configured to some temp execution path. And we have some runtime issues in the logs concerning missing R, but what is R used for in the dbt task?

topic Create a docker image for dbt task in Data Engineering

Create a docker image for dbt task

Re: Create a docker image for dbt task