SQL AI functions in EU region

[drag7ter]

I know that currently foundation model with pay-per-token are not available in EU only in US. In EU I should create serving point and use provisioned foundation model.

But even creating a serving point with llm from catalog (share models). I used the same LLM as AI Funcrions use mixtral_8x7b_instruct

Then after serving point was successfully provisioned I started SQL Serverless Warehouse in the same workspace in EU region, but running sql statements fail with an error.

SELECT ai_analyze_sentiment('I am happy');

[AI_FUNCTION_HTTP_REQUEST_ERROR] Error occurred while making an HTTP request for function `ai_analyze_sentiment`: [REMOTE_FUNCTION_HTTP_FAILED_ERROR] The remote HTTP request failed with code 404, and error message 'HTTP request failed with status: {"error_code":"RESOURCE_DOES_NOT_EXIST","message":"The given endpoint does not exist, please retry after checking the specified model and version deployment exists."}' SQLSTATE: 57012 SQLSTATE: 08000

Could someone tell steps how use ai functions in EU region. Is it possible?

[Kaniz_Fatma]

Hi @drag7ter, Yes, you are correct.

• Pay-per-token Foundation Model APIs are only available in the United States region and are not available in other geographic areas including the EU.
• Provisioned throughput Foundation Model APIs are available in the following regions: United States, Australia, Brazil, Canada, EU Data Boundary, and India.
• So you can use provisioned throughput Foundation Model APIs in the EU Data Boundary region.
• Mosaic AI Model Training, which allows customizing a foundation model using your own data, is only available in the United States region and not in other geographic areas including the EU.
• AI Functions, such as ai_analyze_sentiment, are not available in the EU region. The error message you received suggests the specified endpoint does not exist in your EU region workspace.

In summary, to use foundation models in the EU region, you will need to:

1. Provision a Databricks workspace in the EU Data Boundary region.
2. Use the provisioned throughput Foundation Model APIs to access the curated foundation models, such as Mixtral-8x7B Instruct, in your EU region workspace.
3. For custom model training, you will not be able to use the Mosaic AI Model Training service, and will need to explore other options for fine-tuning foundation models in the EU.
4. The AI Functions feature, including ai_analyze_sentiment, does not appear to be available in the EU region based on the error you encountered. 

The key limitation seems to be the restricted availability of certain foundation model services in the EU compared to the U.S. Provisioned throughput APIs are available, but the pay-per-token and custom training options are not accessible in the EU region currently.