A Theoretical Understanding of Chain-of-Thought

⛓️ A Theoretical Understanding of Chain-of-Thought: Coherent Reasoning and Error-Aware Demonstration

The paper explores Chain-of-Thought (CoT) prompting, a method to enhance the reasoning skills of large language models (LLMs). It introduces Coherent CoT, where reasoning from previous steps is integrated during predictions, leading to better error correction and accuracy compared to a step-by-step approach. The study shows that errors in intermediate reasoning steps have a more significant impact on the final outcome than mistakes in the final response. Based on this, the authors propose an error-aware CoT prompting method, which includes both correct and incorrect reasoning in demonstrations, allowing LLMs to improve reasoning by learning from earlier mistakes.

🔗 Link to paper