Databricks Custom model Serving endpoint Failing

[vaidhaicha]

Hello all,

I have created a custom model serving endpoint in Azure databricks. This endpoint connects with the AzureopenAI model and Azure postgres connection.All of these Azure services are with Private endpoints. When I run this notebook,I am able to make a call, but when I am querying through this serving endpoint using my PAT,its failing. May I know how can I call my Azure resources with endpoint. Is it happening because of personal access token?I am surprised how I can connect to all Azure resources through notebook but not from serving Endpoints?

[Kaniz_Fatma]

Hi @vaidhaicha, It sounds like you’re encountering issues with your custom model serving endpoint in Azure Databricks, specifically when querying through the serving endpoint using your Personal Access Token (PAT).

• A private endpoint is a network interface that uses a private IP address from your virtual network. It securely connects you to a service powered by Azure Private Link.
• By enabling a private endpoint, you bring the service into your virtual network, ensuring private and secure communication.
• Services that can be accessed via private endpoints include Azure Storage, Azure Cosmos DB, and Azur...¹.
• Private endpoints are particularly useful for scenarios where you want to avoid exposure to the public internet.
• PATs are used for authentication and authorization within Azure services.
• When you’re querying your serving endpoint using a PAT, it’s essential to ensure that your private endpoint configuration allows traffic from your virtual network to the specific service.
• Check if your PAT has the necessary permissions to access the service via the private endpoint.
• Verify that your private endpoint is correctly configured:
• Ensure that the private endpoint is associated with the correct service (e.g., AzureopenAI model or Azure Postgres).
• Check the connection status of your private endpoint (e.g., approved, pending, rejected, disconnected).
• Review your PAT permissions:
• Make sure your PAT has the necessary permissions to access the service.
• Check if any role-based access control (RBAC) restrictions apply.

•  

Remember that private endpoints enable connectivity within specific network boundaries, so ensure that your setup aligns with these constraints. If you encounter further issues, consider reaching out for personalized assistance. 😊²³

[vaidhaicha]

But when I create serving endpoint , it is considered as public addresss.We dont have an option to make it private.

So basically from public address(serving endpoint), we are trying to connect to azure services (based on notebook code)on private endpoint which is not allowed. What are the other options can I try to make it work. I want this response generated by notebook to be sent instantly to the front end , so workflows cant be an option as it takes time . How to make sure databricks notebook code can be executed or used by front end application.