Consider moving to chat/completions API

[markmc]

From @gabe-l-hart here

it looks like these prompt templates are largely encoding the chat template for the model. For most models that I've seen, the chat template is actually encoded in the model metadata and applied server-side when using the chat/completions API. Has there been thought around using the chat API rather than applying the template client-side to the lower-level completions API?

[github-actions]

This issue has been automatically marked as stale because it has not had activity within 90 days. It will be automatically closed if no further activity occurs within 30 days.

[bbrowning]

Unstale - some of the research work we're merging in starts moving us over to the chat completions API instead of the generic completions API. So, we will be addressing at least the spirit of this issue.

[bbrowning]

#461 adds an LLMMessagesBlock that uses the chat completions API, as opposed to the LLMBlock in Pipelines that uses the older completions API. However, adding that block is just the first step towards moving us over - we still need to evaluate how/if we'd move our default pipelines over to the chat completions API instead of the completions API.