Instruction Optimization for Tabular Fact Verification

This research article presents a systematic comparison of instruction optimization techniques for enhancing the reasoning performance of large language models (LLMs) in the context of tabular fact verification. The study evaluates four prompting techniques, including direct prediction, Chain-of-Thought (CoT), ReAct with SQL tools, and CodeAct with Python execution, utilizing the DSPy optimization framework. It examines three specific optimizers—COPRO, MiPROv2, and SIMBA—across four benchmarks and two model families. The paper addresses the impacts of instruction optimization on various prompting techniques and analyzes how these optimizers affect the performance of LLMs in verifying claims against structured data. The findings aim to contribute to a better understanding of the effectiveness of instruction optimization in improving the generalization and practical application of LLMs in tabular fact verification tasks. The research underscores the importance of exploring diverse prompting strategies and optimization methods to enhance model performance in this domain.