cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Custom container doesn't launch systemd

HQJaTu
New Contributor III

Quite soon after moving from VMs to containers, I started crafting my own images. That way notebooks have all the necessary libraries already there and no need to do any Pipping/installing in the notebook.

As requirements get more complex, now I'm at the point where I really really need the container to have proper systemd init running.

Databricks Ubuntu 20.04.5 LTS standard container does that, there is systemd with all the bells/whistles that come with that. For obvious reasons, when defining a custom container no init is being run. Instead I have /bin/bash as PID 1.

Any ideas on how to have systemd init and custom container image?

2 REPLIES 2

Debayan
Databricks Employee
Databricks Employee

Hi @Jari Turkia​ , Please check if this helps: https://developers.redhat.com/blog/2019/04/24/how-to-run-systemd-in-a-container#other_cool_features_...

Also, you can run ubuntu /redhat linux OS inside containers which will have systems by default and modify it as per your needs?

HQJaTu
New Contributor III

Databricks uses LXC as container platform. Howevr, the problem is best described in Podman manual page at https://github.com/containers/podman/blob/main/docs/source/markdown/options/systemd.md.

Without LXC doing some ground work first by mounting tmpfs, running systemd init as PID 1 isn't possible. This can easily be achieved on a personal workstation. Doing the same on Databricks isn't possible as I understand it and that's the reason I asked.

To repeat: This functionality exists if choosing to run Databricks provided container.

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group