topic Re: Custom container doesn't launch systemd in Data Engineering

Custom container doesn't launch systemd

HQJaTu — Wed, 02 Nov 2022 16:42:26 GMT

Quite soon after moving from VMs to containers, I started crafting my own images. That way notebooks have all the necessary libraries already there and no need to do any Pipping/installing in the notebook.

As requirements get more complex, now I'm at the point where I really really need the container to have proper systemd init running.

Databricks Ubuntu 20.04.5 LTS standard container does that, there is systemd with all the bells/whistles that come with that. For obvious reasons, when defining a custom container no init is being run. Instead I have /bin/bash as PID 1.

Any ideas on how to have systemd init and custom container image?

Re: Custom container doesn't launch systemd

HQJaTu — Fri, 04 Nov 2022 11:00:53 GMT

Databricks uses LXC as container platform. Howevr, the problem is best described in Podman manual page at https://github.com/containers/podman/blob/main/docs/source/markdown/options/systemd.md.

Without LXC doing some ground work first by mounting tmpfs, running systemd init as PID 1 isn't possible. This can easily be achieved on a personal workstation. Doing the same on Databricks isn't possible as I understand it and that's the reason I asked.

To repeat: This functionality exists if choosing to run Databricks provided container.

Re: Custom container doesn't launch systemd

Debayan — Fri, 04 Nov 2022 08:09:32 GMT

Hi @Jari Turkia , Please check if this helps: https://developers.redhat.com/blog/2019/04/24/how-to-run-systemd-in-a-container#other_cool_features_about_podman_and_systemd

Also, you can run ubuntu /redhat linux OS inside containers which will have systems by default and modify it as per your needs?

Re: Custom container doesn't launch systemd

futurewasfree — Sat, 10 May 2025 12:00:47 GMT

Are there any updates on this? I'm also very interested in having full-fledged Databricks system services integrated into DCS.