How do data roles overlap in your Databricks projects?

[juanlozadab]

The other day, my brother asked me a simple question:

“What does a Data Engineer actually do?”

I started talking about pipelines, databases, and cloud platforms. His face told me that none of it was helping.

So I tried a simpler explanation using a library:

- A Data Architect designs the blueprint: how everything should be organized, connected, governed, and accessed.
- A Data Engineer makes sure the books arrive, are correctly labeled, stored, and available when someone needs them.
- A Data Analyst uses the library to understand what happened and support business decisions.
- A Data Scientist looks for patterns and builds predictions about what may happen next.
- An AI Engineer builds an intelligent librarian that can understand questions, find information, generate answers, and perform tasks.

The analogy worked better, but it also made me think about how unclear these boundaries can become in real projects.

In my experience, most people have a primary role, but the work rarely stays inside that box.

A Data Engineer may contribute to the architecture, investigate a business metric, help with governance, or support an AI use case. In a Databricks project, the same team may be discussing Lakehouse design, building pipelines, validating data in SQL, and helping move an AI application toward production.

I am curious about how other teams organize this:

- Do you have clearly separated roles or broader responsibilities?
- Who owns architecture and governance in your projects?
- Where do Data Engineering and AI Engineering overlap the most?
- Does the division change depending on the size of the team?

I would love to hear how this works in your organization.

[larson096vikky]

Hello, I think the boundaries between these roles often depend on the size of the team and the complexity of the project. In smaller teams, people naturally take on multiple responsibilities, while larger organizations tend to have more specialized roles.

[juanlozadab]

Hi @larson096vikky! 

Totally agree. In my experience, that is one of the reasons smaller companies can be great places to learn: you are constantly pushed beyond your formal role and given opportunities to take on new responsibilities.

Larger organizations can be valuable for a different reason. Roles are often more clearly defined, allowing you to specialize and work within more predictable boundaries.

Neither is necessarily better. It depends on whether, at that point in your career, you are looking for broader challenges or more focus and stability.

[vkondepati]

There are few more roles.

Data Owners hold ultimate accountability for a specific data domain and are responsible for reviewing and approving user access.
Data Stewards manage day-to-day data quality and ensure the right information securely reaches the right people.
Semantic Stewards standardize business terms and KPIs while leading meetings to align cross-departmental definitions.

[juanlozadab · Accepted solution]

Absolutely, @vkondepati  . There are many more roles across the data ecosystem, especially on the governance side, as you mentioned.

My goal was to keep the post focused on some of the most common roles involved in building data products. Even then, titles and responsibilities can vary significantly between companies. Roles such as Analytics Engineer, Data Visualization Developer, ML Engineer, Data Owner, and Data Steward may be separate positions in one organization and shared responsibilities in another.

Thanks for adding this perspective!