Databricks Community

Hi @johndoe99012 If the answer resolved your question, please consider marking it as the solution. It helps others in the community find answers more easily.  

Thanks, I will review it and get back. I'll DIM you.

Thanks for the reply @Walter_C. This didn't quite work, since it used a CPU and didn't consider the max_provisioned_throughput, but I finally got it to work like this: from mlflow.deployments import get_deploy_client client = get_deploy_client("data...

I suspected the issue is coming from this small error I got: Got error: Must specify a chain Type in config. I used the chain_type="stuff" when building the langchain but I'm not sure how to fix it.

You might need to follow steps in https://docs.databricks.com/en/machine-learning/model-serving/route-optimization.html#query-route-optimized-model-serving-endpoints on how to fetch an OAuth token programmatically

If I'm understanding, all you really want to do is have a pre/post - process function running with your model, is that correct? If so, you can do this by using the MLflow pyfunc model. Something  like they do here:https://docs.databricks.com/en/machi...

I finally found it in the docs: https://docs.databricks.com/en/machine-learning/model-serving/inference-tables.html#specify-client_request_id { "client_request_id": "<user-provided-id>", "dataframe_records": [...]}  

Hi, Did you get the solution for this? This timeout issue.

Hi @damselfly20 unfortunately I can't help much with that as I've never worked with RAGs. Are you sure it's the same error though? @NaeemS's and my errors seems to be Java related and yours MLflow related.