Struggling to find clear documentation which can help me with the subject. Need to know all the ways (production best practices) along with API method. As far as I know, using PAT is not a production best practice
Whats the reason behind receiving below error at some rare occassions (once in a month)?"error_code": "BAD_REQUEST", "message": "Encountered an unexpected error while evaluating the model. Verify that the input is compatible with the model for infere...
Is there an inbuilt method to measure latency metrics like TTFT, TBT when deploying agents on Databricks? Using MLFlow ChatAgent, ChatDatabricks/OpenAI client(workspace client)What would be the way to measure them in case no inbuilt method exists?
Code works fine locally but deployment in serving endpoint gives me below error at runtime:{"error_code": "BAD_REQUEST", "message": "Encountered an unexpected error while evaluating the model. Verify that the input is compatible with the model for in...
Thanks for your response Louis. If I understand it correctly, for production monitoring, we would have to rely on client side logging. Can mlflow.log_metric be integrated with traces by any chance? (Since that seems to be the only way to measure TTFT...
