cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best practices. Connect with administrators and architects to optimize your Databricks environment for performance, scalability, and security.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

GitLab on DCS, Datarbricks Container Services

TSK
New Contributor

I would like to set up GitLab and Grafana servers using Databricks Container Services (DCS). The reason is that our development team is small, and the management costs of using EKS are not justifiable. We want to make GitLab and Grafana accessible internally using DCS. Is this possible?

1 REPLY 1

mark_ott
Databricks Employee
Databricks Employee

Yes, it is possible to set up GitLab and Grafana servers using Databricks Container Services (DCS) for internal accessibility. DCS supports custom Docker containers and allows you to deploy server applications such as GitLab and Grafana, making it a viable solution for small development teams seeking to avoid the higher management overhead of EKS or similar orchestrators.โ€‹

GitLab on Databricks Container Services

You can deploy GitLab by running its official Docker image within a DCS-enabled compute environment. The process generally involves:

  • Creating a DCS cluster with a custom Docker image containing GitLab or by pulling the official GitLab Docker image.

  • Configuring network rules so that GitLab is accessible to your internal team.

  • Setting up data persistence using attached storage options available to Databricks containers.โ€‹

GitLab is designed to operate inside Docker, and you can use Compose or Swarm for multi-container setups if your workflow needs runners or additional services.โ€‹

Grafana on Databricks Container Services

Grafana can similarly be deployed as a Docker container inside DCS. You can:

  • Use the official Grafana image and pass configuration parameters as environment variables or mount required configuration and dashboards as volumes.

  • Connect Grafana to Databricks data sources using available plugins (such as the Databricks Grafana datasource).

  • Ensure proper internal network exposure by configuring container networking and firewall rules as needed.โ€‹

Practical Considerations

  • Databricks Container Services focuses on customizing the compute environment. You have full control over containers, but integration, access, and authentication (such as personal access tokens for these services) must be managed according to your security requirements.โ€‹

  • Persistence (for GitLab repositories or Grafana dashboards) should be configured using attached cloud storage or mounted volumes.

  • Internal-only access can be achieved with proper VPC/network settings so only your team can reach these applications.โ€‹

In summary, DCS offers a flexible and cost-effective solution for hosting internal tools like GitLab and Grafana, especially for smaller teams who prefer simplicity over running a full Kubernetes cluster.โ€‹