https://community.databricks.com/t5/technical-blog/how-to-combine-ai-agents-and-audience-segmentation-in-databricks/ba-p/117955 <H3><SPAN>The Problem: What is missing from traditional audience segmentation?</SPAN></H3> <P><SPAN>Audience segmentation has long been crucial for marketing, grouping customers into distinct profiles based on shared demographics, behaviours, or interests. It helps advertisers and ad agencies build personas and create targeted messages. However, classic segmentation doesn’t provide qualitative insight. It tells you </SPAN><I><SPAN>who</SPAN></I><SPAN> your audience is, but now </SPAN><I><SPAN>how</SPAN></I><SPAN> they think, feel, or might react to your creative. It doesn’t give a way to test new ideas, iterate messaging, or provide feedback before launching a campaign. Focus groups and A/B testing are traditional tools to solve for these challenges, but they’re slow, expensive, and hard to scale. Running a focus group for every creative concept isn’t feasible in fast-moving industries.</SPAN></P> <P><SPAN>We can summarise the challenge as: advertisers want to ask questions like: “How would this segment respond to this ad?” and get measured feedback instantly.</SPAN></P> <P><SPAN>In this blog, we’ll show you how to combine </SPAN><STRONG>audience segmentation</STRONG><SPAN> with </SPAN><STRONG>AI agents</STRONG><SPAN> in Databricks to simulate real audience reactions. Think of it as a synthetic focus group powered by customer data and large language models (LLMs), helping marketers make smarter, faster targeting decisions.</SPAN></P> <P>&nbsp;</P> <H3><SPAN>The Solution: Simulating audience reactions with AI Agents in Databricks</SPAN></H3> <P><SPAN>So how do we move beyond classic segmentation and actually interact with personas before launching a campaign?</SPAN></P> <P><SPAN>Here’s a step-by-step breakdown of how to combine segmentation, unstructured data, and LLMs to create an AI-powered synthetic focus group in Databricks (note: you can find the code repo for this solution </SPAN><A href="https://github.com/databricks-solutions/databricks-blogposts/tree/main/2025-04-ai-audience-profiles" target="_blank" rel="noopener"><SPAN>here</SPAN></A><SPAN>).&nbsp;</SPAN></P> <P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="Figure 1: High-level architecture" style="width: 755px;"><img src="https://community.databricks.com/t5/image/serverpage/image-id/16525iA61E90823D3C748D/image-dimensions/755x427?v=v2" width="755" height="427" role="button" title="JackSandom_0-1746547352025.png" alt="Figure 1: High-level architecture" /><span class="lia-inline-image-caption" onclick="event.preventDefault();">Figure 1: High-level architecture</span></span></P> <H4><SPAN>Step 0: Requirements to get started</SPAN></H4> <P><SPAN>You’ll need structured demographic data (one row per customer) and unstructured data like paid product reviews in a </SPAN><A href="https://docs.databricks.com/aws/en/volumes/" target="_blank" rel="noopener"><SPAN>Unity Catalog Volume</SPAN></A><SPAN> (e.g., JSON files). The solution was tested on both serverless and classic compute clusters (15.4 ML runtime).</SPAN></P> <H4><SPAN>Step 1: Use segmentation to identify the different audiences we want to target&nbsp;</SPAN></H4> <P><SPAN>Using structured data (age, income, location), we apply clustering techniques like K-means to group customers into segments.</SPAN></P> <LI-CODE lang="python">k_range = range(2, 8) inertias = [] for k in k_range: kmeans = KMeans(n_clusters=k, random_state=42, n_init="auto") kmeans.fit(X) inertias.append(kmeans.inertia_)</LI-CODE> <P><SPAN>We call each segment a “tribe” which is not just demographic clusters, but groups with shared values and behaviours that influence ad response.</SPAN></P> <P><SPAN>Summary statistics help us to better understand our tribes and create descriptive labels such as “</SPAN><I><SPAN>The Homebodies (Suburban Family-Oriented)</SPAN></I><SPAN>”. Each of these tribes represents a unique blend of demographics, income levels, occupations, and lifestyle preferences. Optionally, </SPAN><A href="https://shap.readthedocs.io/en/latest/" target="_blank" rel="noopener"><SPAN>SHAP</SPAN></A><SPAN> can improve interpretability by revealing feature importance and decision logic behind cluster assignments.</SPAN></P> <P><SPAN>To further bring these tribes to life, we also visualised tribe distribution across an example geographic area (LA neighbourhoods), providing spatial context to behavioural patterns.</SPAN></P> <P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="Figure 2: A Mapbox plot showing the cluster locations" style="width: 671px;"><img src="https://community.databricks.com/t5/image/serverpage/image-id/16527i306730965A63FD18/image-dimensions/671x441?v=v2" width="671" height="441" role="button" title="JackSandom_1-1746547352164.png" alt="Figure 2: A Mapbox plot showing the cluster locations" /><span class="lia-inline-image-caption" onclick="event.preventDefault();">Figure 2: A Mapbox plot showing the cluster locations</span></span></P> <H4><SPAN>Step 2: Enrich each segment with unstructured data to form detailed profiles</SPAN></H4> <P><SPAN>Once we’ve identified tribes, we enrich them with </SPAN><STRONG>unstructured data </STRONG><SPAN>(for example, customer reviews).</SPAN></P> <P><SPAN>We use foundation LLMs in Databricks to summarise this content and generate a </SPAN><STRONG>detailed persona</STRONG><SPAN> that captures interests, tone, sentiment, and pain points in a human-readable way. Each prompt included:</SPAN></P> <UL> <LI style="font-weight: 400;" aria-level="1"><SPAN>Aggregated traits (e.g. median age, income)</SPAN></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>Sample paid reviews for each tribe</SPAN></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>A clear instruction:</SPAN><I><SPAN> “Use the provided demographic information and paid reviews to describe the customer tribe’s characteristics.”</SPAN></I></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>An example format</SPAN></LI> </UL> <LI-CODE lang="python">prompt = """ You are an expert marketing analyst tasked with creating a concise customer persona. Use the provided demographic information and paid reviews to describe the customer tribe’s characteristics. ... The profile should be fluid, easy to read, and written in a professional yet conversational style. """</LI-CODE> <P><SPAN>Databricks allows integration with open-source LLMs (like Llama) or commercial models to generate the personas quickly, repeatably, and scalable. This transforms raw segment data into a rich, narrative description that will help teams better understand not just who the audience is but what matters to them.</SPAN></P> <H4><SPAN>Step 3: Use an AI Agent to simulate audience reactions</SPAN></H4> <P><SPAN>With our enriched personas, we can now create an AI agent that acts as a stand-in for each tribe. You can feed the agent any piece of content (ad copy, campaign messaging) and prompt them to respond in character.</SPAN></P> <P><STRONG>Why an AI agent? At its core, an agent is a system that can reason, respond, and take actions based on data and context. In this case, the agent is built to behave like a specific audience segment,</STRONG><SPAN> using its profile to simulate how a real person in the segment might respond to ad content.</SPAN></P> <P><SPAN>Rather than just returning one-off completions like a basic LLM call, the agent can:</SPAN></P> <UL> <LI style="font-weight: 400;" aria-level="1"><SPAN>Stay “in character” based on the persona description</SPAN></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>Interpret and respond to user inputs</SPAN></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>Access external tools, such as retrieving previous ad campaigns (RAG)</SPAN></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>Stay focused on the task of helping the user improve their ad content</SPAN></LI> </UL> <P><SPAN>This is what turns the static personas into a </SPAN><STRONG>synthetic focus group</STRONG><SPAN>.&nbsp;</SPAN></P> <H5><FONT color="#808080"><I><SPAN>Building the agent in Databricks</SPAN></I></FONT></H5> <P><SPAN>We built this agent using LangChain and LangGraph open source frameworks. The Genai agent was deployed as an API with MLflow&nbsp; and Databricks model serving and the LLM was a Databricks-hosted LLM,open source Llama 3.3 in this case, chosen for its advanced&nbsp; reasoning capabilities.</SPAN></P> <H5><STRONG>LLM + Dynamic System Prompt</STRONG></H5> <P><SPAN>Each agent starts with a system prompt dynamically constructed from:</SPAN></P> <UL> <LI style="font-weight: 400;" aria-level="1"><SPAN>The audience tribe (e.g. “Tech-Savvy Professional”)</SPAN></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>The pre-created persona profile</SPAN></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>Optional retrieved ad content via RAG (see below)</SPAN></LI> </UL> <P><SPAN>The system prompt ensures the model stays in character and provides feedback only from the audience’s perspective.</SPAN></P> <LI-CODE lang="python">system_prompt = PromptTemplate( input_variables=["tribe", "profile", "retrieved_ads"], template=""" You are an audience persona named {tribe} with the following profile: {profile} The user is an advertising content writer and wants to tailor copy specific to your persona. Your goal is to assist the user in doing this by acting as a {tribe} and helping the user to test ideas and get to tailored ad content which is effective on your persona. Here is a past successful ad for this tribe: {retrieved_ads} Use this ad as inspiration if it is relevant to the user's query. If it is not relevant, ignore. If prompted to improve or generate new ad content, always provide suggested copy. Always end by asking a question or offering a suggestion to help the user get to their goal. Stay in character always and only respond in the context of your audience persona. Do not make stuff up. If asked about something unrelated, politely redirect the conversation. """ ) formatted_prompt = system_prompt.format( tribe=tribe, profile=profile, retrieved_ads=retrieved_ads_text )</LI-CODE> <P><BR /><STRONG>RAG (Retrieval-Augmented Generation) with Vector Search</STRONG></P> <P><SPAN>To make agent feedback more realistic and context-aware, we used RAG to reference prior successful campaigns. This can also be extended to include brand guidelines or other context.</SPAN></P> <P><SPAN>The agent uses a vector similarity search via </SPAN><SPAN>VectorSearchRetrieverTool</SPAN><SPAN> to pull ad copy from a Databricks Vector Search index. Retrieved content is embedded into the system prompt as optional context.</SPAN></P> <LI-CODE lang="python">vs_tool = VectorSearchRetrieverTool( index_name=VS_INDEX_NAME, columns=["campaign_id", "ad_copy"], tool_name="Ad-Copy-Retriever", tool_description="Retrieve prior successful ad copy for tribe", filters={"tribe": None}, # Placeholder for dynamic filtering ) </LI-CODE> <P><SPAN>This tool is invoked only when relevant (determined by the agent). This is because the tool is dependent on the prompt. For example, if the user asks: </SPAN><I><SPAN>“Help me improve this ad content…”</SPAN></I><SPAN>, the agent will use the tool to retrieve ad copy to help guide the response. But if the user asks </SPAN><I><SPAN>“What do you look for in a product?”</SPAN></I><SPAN>, then the tool is not required.</SPAN></P> <H5><STRONG>LangGraph for Agent Logic</STRONG></H5> <P><SPAN>LangGraph allows us to define agent workflows as stateful graphs. In this case, our workflow is simple:</SPAN></P> <UL> <LI style="font-weight: 400;" aria-level="1"><SPAN>Accept a user message</SPAN></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>Construct a system prompt dynamically</SPAN></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>Optionally retrieve reference ad copy</SPAN></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>Return a model-generated response</SPAN></LI> </UL> <P><SPAN>Because the logic is modular and extendable, we could easily add steps later, such as tone analysis or multi-agent debate.</SPAN></P> <H5><STRONG>Serving via MLflow + Databricks Model Serving</STRONG></H5> <P><SPAN>Finally, the full agent is packaged as a custom </SPAN><I><SPAN>mlflow.pyfunc.ChatAgent</SPAN></I><SPAN> and deployed with Databricks Model Serving, enabling:</SPAN></P> <UL> <LI style="font-weight: 400;" aria-level="1"><SPAN>Streaming conversational responses</SPAN></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>Artifact loading (e.g., </SPAN><I><SPAN>profiles.json</SPAN></I><SPAN> for personas)</SPAN></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>Custom inputs for selecting the tribes on-the-fly *</SPAN></LI> </UL> <P><SPAN>* The use of</SPAN><I><SPAN> custom_inputs</SPAN></I><SPAN> is a powerful aspect of the agent setup as it allows the user to pass dynamic values (like the audience tribe) directly into the agent at runtime without needing to hard-code logic for each persona.</SPAN></P> <LI-CODE lang="python">input_example = { "messages": [{"role": "user", "content": "How can I improve this ad?"}], "custom_inputs": {"tribe": "The Innovators (Tech-Savvy Professional)"}, } AGENT.predict(input_example)</LI-CODE> <P><SPAN>Databricks makes this seamless, offering scalable deployment, unified governance, and a direct path from prototype to production. With this setup, you can interact with each persona as if they were real, transforming static audience documents into interactive, intelligent agents with the ability to supercharge the creative process for advertisers.</SPAN></P> <H4><SPAN>Step 4: Build an interactive front-end with Databricks Apps</SPAN></H4> <P><SPAN>Finally, we make all of this usable for advertisers with a lightweight front-end. Using Databricks Apps, you can build a simple UI that allows users to:</SPAN></P> <UL> <LI style="font-weight: 400;" aria-level="1"><SPAN>Upload ad content</SPAN></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>Choose an audience tribe</SPAN></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>View the AI-generated response and sentiment</SPAN></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>Iterate and refine based on feedback</SPAN></LI> </UL> <P><SPAN>Databricks Apps enables teams to build and deploy this simple front-end without leaving the notebook environment. It seamlessly integrates with the model serving endpoint exposing your agent. There is a choice of Python frameworks such as Flask or Streamlit. In this case, we used Streamlit and customised the pre-built template to generate the UI.</SPAN></P> <LI-CODE lang="python"># Avatars human_avatar = "<span class="lia-unicode-emoji" title=":speech_balloon:">💬</span>" tribe_avatars = { "The Homebodies (Suburban Family-Oriented)": "🧑🏼‍<span class="lia-unicode-emoji" title=":baby_bottle:">🍼</span>", "The Luxe Lifers (High-Income Empty Nester)": "🏌🏻‍<span class="lia-unicode-emoji" title=":female_sign:">♀️</span>", "The Campus Creatives (College Student)": "<span class="lia-unicode-emoji" title=":man_student:">👨🏽‍🎓</span>", "The Quiet Seekers (Retired Rural Dweller)": "<span class="lia-unicode-emoji" title=":man_farmer:">👨🏼‍🌾</span>", "The Innovators (Tech-Savvy Professional)": "<span class="lia-unicode-emoji" title=":woman_technologist:">👩🏼‍💻</span>" } ... # Display assistant response in chat message container with st.chat_message("assistant", avatar=tribe_avatars[st.session_state.selected_tribe]): # Query the Databricks serving endpoint try: response = client.predict( endpoint=os.getenv("SERVING_ENDPOINT"), inputs={ "messages": [{"role": msg["role"], "content": msg["content"]} for msg in st.session_state.messages], "custom_inputs": {"tribe": st.session_state.selected_tribe}, }, ) assistant_response = response['messages'][0]['content']</LI-CODE> <H3>&nbsp;</H3> <H3><SPAN>End-to-end Example</SPAN></H3> <P><SPAN>To bring everything together, let’s walk through a complete example using a fictional audience tribe:&nbsp;</SPAN></P> <P><STRONG>The Innovators (Tech-Savvy Professional) </STRONG><SPAN><span class="lia-unicode-emoji" title=":woman_technologist:">👩🏼‍💻</span></SPAN><SPAN>.</SPAN></P> <P><SPAN>This group was identified through the clustering of demographic data. It consists primarily of younger individuals, living in urban areas, with higher incomes, college education, and no dependents.</SPAN></P> <H4><SPAN>Step 1: Paid reviews</SPAN></H4> <P><SPAN>We generated fake paid reviews (with another LLM) associated with the customers in this tribe.</SPAN></P> <P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="Figure 3: Example paid review stored in JSON format" style="width: 698px;"><img src="https://community.databricks.com/t5/image/serverpage/image-id/16526i67CA99832538BD53/image-dimensions/698x280?v=v2" width="698" height="280" role="button" title="JackSandom_2-1746547352019.png" alt="Figure 3: Example paid review stored in JSON format" /><span class="lia-inline-image-caption" onclick="event.preventDefault();">Figure 3: Example paid review stored in JSON format</span></span></P> <H4><SPAN>Step 2: Persona generated by the LLM</SPAN></H4> <P><SPAN>Using the cluster traits and sample review content, we prompted the LLM to generate a rich, natural-language persona. This acts as a pre-generated profile for the agent.</SPAN></P> <P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="Figure 4: The persona generated from the LLM" style="width: 743px;"><img src="https://community.databricks.com/t5/image/serverpage/image-id/16528iEFFD812D00B79AFE/image-size/large?v=v2&amp;px=999" role="button" title="JackSandom_3-1746547351922.png" alt="Figure 4: The persona generated from the LLM" /><span class="lia-inline-image-caption" onclick="event.preventDefault();">Figure 4: The persona generated from the LLM</span></span></P> <H4><SPAN>Step 3: Using the app to test ad copy</SPAN></H4> <P><SPAN>We then use the Databricks App to upload a piece of content and simulate how “Tech-Savvy Professionals” would respond.</SPAN></P> <P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="Figure 5: Final Databricks App UI" style="width: 483px;"><img src="https://community.databricks.com/t5/image/serverpage/image-id/16534iE3CD857392363361/image-dimensions/483x526?v=v2" width="483" height="526" role="button" title="JackSandom_0-1746550190215.gif" alt="Figure 5: Final Databricks App UI" /><span class="lia-inline-image-caption" onclick="event.preventDefault();">Figure 5: Final Databricks App UI</span></span></P> <P><SPAN>This immediate, persona-specific feedback helps copywriters and marketers iterate quickly but also in a data-driven way as it combines real customer structured + unstructured data with AI reasoning.</SPAN></P> <P>&nbsp;</P> <H3><SPAN>What’s Next: Future Improvements</SPAN></H3> <P><SPAN>This blog lays the groundwork for AI-driven audience insight, but there are opportunities to take it even further:</SPAN></P> <UL> <LI style="font-weight: 400;" aria-level="1"><SPAN>Beyond text: Incorporate image and video inputs using multimodal models to evaluate all kinds of creative content (not just copy)</SPAN></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>Multi-agent interaction: Simulate conversations or debates between audience segments to discover alignment or conflict in perceptions</SPAN></LI> <LI style="font-weight: 400;" aria-level="1"><SPAN>Real-time data integration: Connect to live campaign performance and/or leverage API calling tools to incorporate social media trends or current events for real-world context</SPAN></LI> </UL> <H3>&nbsp;</H3> <H3><SPAN>Conclusion</SPAN></H3> <P><SPAN>In this blog, we showed how combining audience segmentation, LLMs, and agentic AI can simulate audience reactions, creating a synthetic focus group to help advertisers test and refine content. By clustering structured data and enriching it with unstructured insights, we created realistic personas and deployed agents to evaluate ad copy.</SPAN></P> <P><SPAN>The result is a scalable, intelligent alternative to traditional focus groups that is faster to iterate and more cost-effective. Marketers and ad agencies can experiment with content and instantly see how different segments might respond, iterate confidently, and refine messaging before launch.</SPAN></P> <P><SPAN>Databricks enables this with integrated tools for machine learning (MLflow), foundation LLMs, vector search, and Databricks Apps, supporting a complete AI workflow on one unified platform. It’s a powerful example of using Databricks not just for data and analytics, but for building innovative, AI-driven approaches to creativity and ad targeting.</SPAN></P> Mon, 12 May 2025 12:47:13 GMT JackSandom 2025-05-12T12:47:13Z https://community.databricks.com/t5/technical-blog/how-to-combine-ai-agents-and-audience-segmentation-in-databricks/ba-p/117955 <P><span class="lia-inline-image-display-wrapper lia-image-align-inline" image-alt="JackSandom_1-1746549313820.png" style="width: 708px;"><img src="https://community.databricks.com/t5/image/serverpage/image-id/16532i3105BEB4C0291E16/image-dimensions/708x340?v=v2" width="708" height="340" role="button" title="JackSandom_1-1746549313820.png" alt="JackSandom_1-1746549313820.png" /></span></P> <P>&nbsp;</P> <P>&nbsp;</P> <P>&nbsp;</P> Mon, 12 May 2025 12:47:13 GMT https://community.databricks.com/t5/technical-blog/how-to-combine-ai-agents-and-audience-segmentation-in-databricks/ba-p/117955 JackSandom 2025-05-12T12:47:13Z