RARe: Retrieval Augmented Retrieval with In-Context Examples

This document is a technical report that investigates the efficacy of in-context examples in enhancing the performance of embedding models for retrieval tasks. The proposed method, named RARe, fine-tunes a pre-trained model by utilizing in-context examples that are semantically similar to the target queries. The document outlines how standard methods of prepending in-context examples to queries do not work effectively without modification. It presents a systematic approach to augment retrieval models by integrating these examples, ultimately achieving notable performance improvements across several open-domain retrieval datasets. The report details the experimental setup, including training methods and evaluation procedures, and provides comprehensive performance metrics that indicate RARe's superiority in both standard and reasoning-oriented retrieval tasks. Additionally, it includes insights into how the quality and selection of in-context examples influence the model's effectiveness, thereby contextualizing the performance gains observed during experiments.