Hi @askaditya,
Yes, this is possible. You can connect Microsoft Copilot Studio to a Databricks Genie space on AWS by using the Genie Conversation API and Copilot Studio's HTTP Request node (or a Power Automate cloud flow). Here is how the pieces fit together.
OVERVIEW
Databricks exposes a set of REST API endpoints for Genie spaces that let you start conversations, send natural-language questions, and retrieve generated SQL results programmatically. Copilot Studio can call these endpoints using either the built-in HTTP Request node or a Power Automate cloud flow with HTTP actions.
GENIE CONVERSATION API ENDPOINTS
The key endpoints you will use are:
1) Start a conversation
POST https://<your-workspace-url>/api/2.0/genie/spaces/<space_id>/start-conversation
Request body:
{
"content": "What were total sales last quarter?"
}
This returns a conversation_id and message_id along with the Genie response.
2) Send a follow-up message
POST https://<your-workspace-url>/api/2.0/genie/spaces/<space_id>/conversations/<conversation_id>/messages
Request body:
{
"content": "Break that down by region"
}
3) Get a message (check status and retrieve the response)
GET https://<your-workspace-url>/api/2.0/genie/spaces/<space_id>/conversations/<conversation_id>/messages/<message_id>
4) Get query results for an attachment
GET https://<your-workspace-url>/api/2.0/genie/spaces/<space_id>/conversations/<conversation_id>/messages/<message_id>/attachments/<attachment_id>/query-result
Replace <your-workspace-url> with your AWS Databricks workspace URL (for example, dbc-a1b2345c-d6e7.cloud.databricks.com) and <space_id> with the ID of your Genie space, which you can find in the URL when you open the space in the Databricks UI.
AUTHENTICATION
The Genie API uses the standard Databricks authentication. For this integration you have two main options:
a) Personal Access Token (PAT): Generate a PAT in your Databricks workspace (Settings > Developer > Access tokens). Then include it in your HTTP requests as a header:
Authorization: Bearer <your-pat-token>
b) OAuth (recommended for production): Use OAuth with a service principal for stronger security. This requires setting up a service principal in your Databricks workspace, creating OAuth credentials, and then using the token endpoint to obtain an access token before calling the Genie API.
For a quick proof of concept, a PAT is the simplest path. For production use, consider a service principal with OAuth.
COPILOT STUDIO INTEGRATION APPROACH
Option 1: HTTP Request Node (simpler)
In your Copilot Studio topic, use the HTTP Request node:
- Add node > Advanced > Send HTTP request
- Set the URL to the start-conversation endpoint
- Set Method to POST
- Add headers:
Authorization: Bearer <your-token>
Content-Type: application/json
- Set the body to JSON Content with the user's question
- Parse the response to extract the conversation_id, message_id, and the Genie answer
One thing to be aware of: the Genie API is asynchronous. The start-conversation call may return before the response is fully generated. You may need to poll the get-message endpoint until the message status is COMPLETED. In Copilot Studio, you can handle this with a loop using condition nodes that check the status and call the get-message endpoint again after a short delay.
Option 2: Power Automate Cloud Flow (more robust)
Create a Power Automate cloud flow that:
1. Accepts the user question as input
2. Calls the Genie start-conversation endpoint via the HTTP action
3. Polls the get-message endpoint in a Do Until loop until status is COMPLETED
4. Extracts the response text and/or query results
5. Returns the answer to Copilot Studio
Then call this flow from your Copilot Studio topic. This approach gives you better control over the polling logic and error handling, and keeps the Copilot Studio authoring canvas cleaner.
IMPORTANT CONSIDERATIONS
- Network connectivity: Since your Databricks workspace is on AWS, ensure that outbound HTTP calls from Copilot Studio (which runs in Azure/Microsoft cloud) can reach your Databricks workspace URL. If your workspace uses IP access lists or private networking, you will need to allow traffic from Microsoft's IP ranges or set up appropriate network routing.
- Genie space setup: Make sure your Genie space is configured with the right tables, instructions, and example SQL queries so it can answer the types of questions your Copilot Studio users will ask. The quality of the Genie responses depends heavily on how well the space is curated.
- Permissions: The PAT or service principal you use must have access to the Genie space and the underlying Unity Catalog tables.
- Response timeout: Copilot Studio HTTP Request nodes have a default 30-second timeout. Genie responses can sometimes take longer, especially for complex queries. The Power Automate approach handles this better since cloud flows can run for several minutes.
DOCUMENTATION REFERENCES
- Genie API (Databricks REST API reference):
https://docs.databricks.com/api/workspace/genie
- AI/BI Genie spaces overview:
https://docs.databricks.com/en/genie/
- Databricks personal access tokens:
https://docs.databricks.com/en/dev-tools/auth/pat.html
- Copilot Studio HTTP Request node:
https://learn.microsoft.com/en-us/microsoft-copilot-studio/authoring-http-node
- Databricks Python SDK (includes Genie client):
https://github.com/databricks/databricks-sdk-py
Hope this helps you get started with the integration. If you run into any specific issues during setup, feel free to post back with the details.
* This reply used an agent system I built to research and draft this response based on the wide set of documentation I have available and previous memory. I personally review the draft for any obvious issues and for monitoring system reliability and update it when I detect any drift, but there is still a small chance that something is inaccurate, especially if you are experimenting with brand new features.
