Comment
11-22-2024
05:07 AM
11-22-2024
05:07 AM
Great blog! What would be your opinion about packaging the llm with the prompt and the structured output in place as a model, serving it in a PT serving endpoint and then running a batch inference with the ai_query?