The Problem: What is missing from traditional audience segmentation?
Audience segmentation has long been crucial for marketing, grouping customers into distinct profiles based on shared demographics, behaviours, or interests. It helps advertisers and ad agencies build personas and create targeted messages. However, classic segmentation doesn’t provide qualitative insight. It tells you who your audience is, but now how they think, feel, or might react to your creative. It doesn’t give a way to test new ideas, iterate messaging, or provide feedback before launching a campaign. Focus groups and A/B testing are traditional tools to solve for these challenges, but they’re slow, expensive, and hard to scale. Running a focus group for every creative concept isn’t feasible in fast-moving industries.
We can summarise the challenge as: advertisers want to ask questions like: “How would this segment respond to this ad?” and get measured feedback instantly.
In this blog, we’ll show you how to combine audience segmentation with AI agents in Databricks to simulate real audience reactions. Think of it as a synthetic focus group powered by customer data and large language models (LLMs), helping marketers make smarter, faster targeting decisions.
The Solution: Simulating audience reactions with AI Agents in Databricks
So how do we move beyond classic segmentation and actually interact with personas before launching a campaign?
Here’s a step-by-step breakdown of how to combine segmentation, unstructured data, and LLMs to create an AI-powered synthetic focus group in Databricks (note: you can find the code repo for this solution here).
Step 0: Requirements to get started
You’ll need structured demographic data (one row per customer) and unstructured data like paid product reviews in a Unity Catalog Volume (e.g., JSON files). The solution was tested on both serverless and classic compute clusters (15.4 ML runtime).
Step 1: Use segmentation to identify the different audiences we want to target
Using structured data (age, income, location), we apply clustering techniques like K-means to group customers into segments.
k_range = range(2, 8)
inertias = []
for k in k_range:
kmeans = KMeans(n_clusters=k, random_state=42, n_init="auto")
kmeans.fit(X)
inertias.append(kmeans.inertia_)We call each segment a “tribe” which is not just demographic clusters, but groups with shared values and behaviours that influence ad response.
Summary statistics help us to better understand our tribes and create descriptive labels such as “The Homebodies (Suburban Family-Oriented)”. Each of these tribes represents a unique blend of demographics, income levels, occupations, and lifestyle preferences. Optionally, SHAP can improve interpretability by revealing feature importance and decision logic behind cluster assignments.
To further bring these tribes to life, we also visualised tribe distribution across an example geographic area (LA neighbourhoods), providing spatial context to behavioural patterns.
Step 2: Enrich each segment with unstructured data to form detailed profiles
Once we’ve identified tribes, we enrich them with unstructured data (for example, customer reviews).
We use foundation LLMs in Databricks to summarise this content and generate a detailed persona that captures interests, tone, sentiment, and pain points in a human-readable way. Each prompt included:
- Aggregated traits (e.g. median age, income)
- Sample paid reviews for each tribe
- A clear instruction: “Use the provided demographic information and paid reviews to describe the customer tribe’s characteristics.”
- An example format
prompt = """
You are an expert marketing analyst tasked with creating a concise customer persona. Use the provided demographic information and paid reviews to describe the customer tribe’s characteristics.
...
The profile should be fluid, easy to read, and written in a professional yet conversational style.
"""Databricks allows integration with open-source LLMs (like Llama) or commercial models to generate the personas quickly, repeatably, and scalable. This transforms raw segment data into a rich, narrative description that will help teams better understand not just who the audience is but what matters to them.
Step 3: Use an AI Agent to simulate audience reactions
With our enriched personas, we can now create an AI agent that acts as a stand-in for each tribe. You can feed the agent any piece of content (ad copy, campaign messaging) and prompt them to respond in character.
Why an AI agent? At its core, an agent is a system that can reason, respond, and take actions based on data and context. In this case, the agent is built to behave like a specific audience segment, using its profile to simulate how a real person in the segment might respond to ad content.
Rather than just returning one-off completions like a basic LLM call, the agent can:
- Stay “in character” based on the persona description
- Interpret and respond to user inputs
- Access external tools, such as retrieving previous ad campaigns (RAG)
- Stay focused on the task of helping the user improve their ad content
This is what turns the static personas into a synthetic focus group.
Building the agent in Databricks
We built this agent using LangChain and LangGraph open source frameworks. The Genai agent was deployed as an API with MLflow and Databricks model serving and the LLM was a Databricks-hosted LLM,open source Llama 3.3 in this case, chosen for its advanced reasoning capabilities.
LLM + Dynamic System Prompt
Each agent starts with a system prompt dynamically constructed from:
- The audience tribe (e.g. “Tech-Savvy Professional”)
- The pre-created persona profile
- Optional retrieved ad content via RAG (see below)
The system prompt ensures the model stays in character and provides feedback only from the audience’s perspective.
system_prompt = PromptTemplate(
input_variables=["tribe", "profile", "retrieved_ads"],
template="""
You are an audience persona named {tribe} with the following profile:
{profile}
The user is an advertising content writer and wants to tailor copy specific to your
persona. Your goal is to assist the user in doing this by acting as a {tribe} and
helping the user to test ideas and get to tailored ad content which is effective on
your persona.
Here is a past successful ad for this tribe:
{retrieved_ads}
Use this ad as inspiration if it is relevant to the user's query. If it is not
relevant, ignore.
If prompted to improve or generate new ad content, always provide suggested copy.
Always end by asking a question or offering a suggestion to help the user get to
their goal.
Stay in character always and only respond in the context of your audience persona.
Do not make stuff up. If asked about something unrelated, politely redirect the
conversation.
"""
)
formatted_prompt = system_prompt.format(
tribe=tribe,
profile=profile,
retrieved_ads=retrieved_ads_text
)
RAG (Retrieval-Augmented Generation) with Vector Search
To make agent feedback more realistic and context-aware, we used RAG to reference prior successful campaigns. This can also be extended to include brand guidelines or other context.
The agent uses a vector similarity search via VectorSearchRetrieverTool to pull ad copy from a Databricks Vector Search index. Retrieved content is embedded into the system prompt as optional context.
vs_tool = VectorSearchRetrieverTool(
index_name=VS_INDEX_NAME,
columns=["campaign_id", "ad_copy"],
tool_name="Ad-Copy-Retriever",
tool_description="Retrieve prior successful ad copy for tribe",
filters={"tribe": None}, # Placeholder for dynamic filtering
)
This tool is invoked only when relevant (determined by the agent). This is because the tool is dependent on the prompt. For example, if the user asks: “Help me improve this ad content…”, the agent will use the tool to retrieve ad copy to help guide the response. But if the user asks “What do you look for in a product?”, then the tool is not required.
LangGraph for Agent Logic
LangGraph allows us to define agent workflows as stateful graphs. In this case, our workflow is simple:
- Accept a user message
- Construct a system prompt dynamically
- Optionally retrieve reference ad copy
- Return a model-generated response
Because the logic is modular and extendable, we could easily add steps later, such as tone analysis or multi-agent debate.
Serving via MLflow + Databricks Model Serving
Finally, the full agent is packaged as a custom mlflow.pyfunc.ChatAgent and deployed with Databricks Model Serving, enabling:
- Streaming conversational responses
- Artifact loading (e.g., profiles.json for personas)
- Custom inputs for selecting the tribes on-the-fly *
* The use of custom_inputs is a powerful aspect of the agent setup as it allows the user to pass dynamic values (like the audience tribe) directly into the agent at runtime without needing to hard-code logic for each persona.
input_example = {
"messages": [{"role": "user", "content": "How can I improve this ad?"}],
"custom_inputs": {"tribe": "The Innovators (Tech-Savvy Professional)"},
}
AGENT.predict(input_example)Databricks makes this seamless, offering scalable deployment, unified governance, and a direct path from prototype to production. With this setup, you can interact with each persona as if they were real, transforming static audience documents into interactive, intelligent agents with the ability to supercharge the creative process for advertisers.
Step 4: Build an interactive front-end with Databricks Apps
Finally, we make all of this usable for advertisers with a lightweight front-end. Using Databricks Apps, you can build a simple UI that allows users to:
- Upload ad content
- Choose an audience tribe
- View the AI-generated response and sentiment
- Iterate and refine based on feedback
Databricks Apps enables teams to build and deploy this simple front-end without leaving the notebook environment. It seamlessly integrates with the model serving endpoint exposing your agent. There is a choice of Python frameworks such as Flask or Streamlit. In this case, we used Streamlit and customised the pre-built template to generate the UI.
# Avatars
human_avatar = "💬"
tribe_avatars = {
"The Homebodies (Suburban Family-Oriented)": "🧑🏼🍼",
"The Luxe Lifers (High-Income Empty Nester)": "🏌🏻♀️",
"The Campus Creatives (College Student)": "👨🏽🎓",
"The Quiet Seekers (Retired Rural Dweller)": "👨🏼🌾",
"The Innovators (Tech-Savvy Professional)": "👩🏼💻"
}
...
# Display assistant response in chat message container
with st.chat_message("assistant", avatar=tribe_avatars[st.session_state.selected_tribe]):
# Query the Databricks serving endpoint
try:
response = client.predict(
endpoint=os.getenv("SERVING_ENDPOINT"),
inputs={
"messages": [{"role": msg["role"], "content": msg["content"]} for msg in st.session_state.messages],
"custom_inputs": {"tribe": st.session_state.selected_tribe},
},
)
assistant_response = response['messages'][0]['content']End-to-end Example
To bring everything together, let’s walk through a complete example using a fictional audience tribe:
The Innovators (Tech-Savvy Professional) 👩🏼💻.
This group was identified through the clustering of demographic data. It consists primarily of younger individuals, living in urban areas, with higher incomes, college education, and no dependents.
Step 1: Paid reviews
We generated fake paid reviews (with another LLM) associated with the customers in this tribe.
Step 2: Persona generated by the LLM
Using the cluster traits and sample review content, we prompted the LLM to generate a rich, natural-language persona. This acts as a pre-generated profile for the agent.
Step 3: Using the app to test ad copy
We then use the Databricks App to upload a piece of content and simulate how “Tech-Savvy Professionals” would respond.
This immediate, persona-specific feedback helps copywriters and marketers iterate quickly but also in a data-driven way as it combines real customer structured + unstructured data with AI reasoning.
What’s Next: Future Improvements
This blog lays the groundwork for AI-driven audience insight, but there are opportunities to take it even further:
- Beyond text: Incorporate image and video inputs using multimodal models to evaluate all kinds of creative content (not just copy)
- Multi-agent interaction: Simulate conversations or debates between audience segments to discover alignment or conflict in perceptions
- Real-time data integration: Connect to live campaign performance and/or leverage API calling tools to incorporate social media trends or current events for real-world context
Conclusion
In this blog, we showed how combining audience segmentation, LLMs, and agentic AI can simulate audience reactions, creating a synthetic focus group to help advertisers test and refine content. By clustering structured data and enriching it with unstructured insights, we created realistic personas and deployed agents to evaluate ad copy.
The result is a scalable, intelligent alternative to traditional focus groups that is faster to iterate and more cost-effective. Marketers and ad agencies can experiment with content and instantly see how different segments might respond, iterate confidently, and refine messaging before launch.
Databricks enables this with integrated tools for machine learning (MLflow), foundation LLMs, vector search, and Databricks Apps, supporting a complete AI workflow on one unified platform. It’s a powerful example of using Databricks not just for data and analytics, but for building innovative, AI-driven approaches to creativity and ad targeting.
