Summary
This technical report details the Claire project, which investigates the potential of small language models (LLMs) to perform causal reasoning through supervised fine-tuning (SFT) with synthetic data. The project aims to create novel intellectual property that may inspire future product development and attract interest from venture capitalists. The methodology includes generating synthetic training data using GPT-4.1 and fine-tuning three base LLMs with Low-Rank Adaptation (LoRA) to reduce training costs. Key results indicate that the Claire-1.5B and Claire-7B models outperformed their base counterparts in causal reasoning tasks, particularly on GSM8K and GPQA benchmarks. The report also discusses limitations such as generalization issues and the need for a more diverse dataset. Next steps involve enhancing training data, improving prompt design, and exploring deployment strategies. Overall, the project represents a significant advancement towards developing efficient, causality-aware LLMs.
