Composable scaffolding for CoT

[dhruviyer]

Hello all,

As part of building toward #50 I wanted to lay some ground work for working with CoT prompts and completions. As a start I implemented this for sem_filter and want to get some comments/feedback on the design before rolling it out to other operators.

At a high level, I am proposing a few changes:

1. We use XML to cleanly separate reasoning sections and answer sections on the model outputs, which I hope will make generation and parsing more reliable and also can be reused across all the various semantic operators.

Hence, the PR revamps the way the model prompt to require the model to fill out this template:

<Reasoning> Provide your reasoning here. {customer reasoning instructions} </Reasoning> <Answer> Provide your answer here. {custom answer instructions} </Answer>

Correspondingly, it also formats the CoT examples in this same template when provided by the user

2. The PR also simplifies the logic around (1) prompting the model to do CoT and (2) providing examples of CoT. The following configurations are now implicitly handled in the single filter_formatter() function:

• No CoT prompting, no examples
• CoT prompting, no CoT examples
• CoT prompting, CoT examples
• No CoT prompting, CoT examples

As mentioned, mostly want to get feedback from the authors if this approach makes sense

[dhruviyer]

@liana313 after our discussion I removed the formatting of CoT using XML. This PR now introduces a standardized way of creating CoT prompts and parsing CoT responses. Following PRs will expand support to other semantic operators and implement retries (whichever PR solves #43)