cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Custom container doesn't launch systemd

HQJaTu
New Contributor III

Quite soon after moving from VMs to containers, I started crafting my own images. That way notebooks have all the necessary libraries already there and no need to do any Pipping/installing in the notebook.

As requirements get more complex, now I'm at the point where I really really need the container to have proper systemd init running.

Databricks Ubuntu 20.04.5 LTS standard container does that, there is systemd with all the bells/whistles that come with that. For obvious reasons, when defining a custom container no init is being run. Instead I have /bin/bash as PID 1.

Any ideas on how to have systemd init and custom container image?

2 REPLIES 2

Debayan
Esteemed Contributor III
Esteemed Contributor III

Hi @Jari Turkia​ , Please check if this helps: https://developers.redhat.com/blog/2019/04/24/how-to-run-systemd-in-a-container#other_cool_features_...

Also, you can run ubuntu /redhat linux OS inside containers which will have systems by default and modify it as per your needs?

HQJaTu
New Contributor III

Databricks uses LXC as container platform. Howevr, the problem is best described in Podman manual page at https://github.com/containers/podman/blob/main/docs/source/markdown/options/systemd.md.

Without LXC doing some ground work first by mounting tmpfs, running systemd init as PID 1 isn't possible. This can easily be achieved on a personal workstation. Doing the same on Databricks isn't possible as I understand it and that's the reason I asked.

To repeat: This functionality exists if choosing to run Databricks provided container.

Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!