Performance Comparison of Large Language Models in Insurance

This report presents a performance comparison of six different Large Language Models (LLMs) applied to common processes in the insurance industry. The objective is to evaluate how these models can enhance the efficiency and accuracy of underwriting, claims, and risk processes. The LLMs tested include GPT3.5, GPT4, Mistral Large, Llama2-70B, Llama2-13B, and Llama2-7B, across various scenarios such as information extraction from airline invoices, property repair quotes, and dental invoices. The findings indicate that LLMs with larger context sizes generally perform better, although cost considerations must be taken into account. Effective prompt engineering is also highlighted as crucial for optimizing performance. The report aims to provide insurance professionals with reliable information to make informed decisions regarding the integration of Generative AI into their operations.