Comparing LLMs for Prompt-Enhanced ACT-R and Soar Model Development

This case study presents experiments focused on the application of ChatGPT4 and Google Bard in the development of ACT-R and Soar cognitive models. The document outlines two cognitive tasks where these Large Language Models (LLMs) serve as conversational interfaces within the model development environments. The first task involves creating an intelligent driving model using ACT-R, which incorporates motor and perceptual behaviors, allowing interaction with an unmodified interface. The second task assesses the development of educational skills using the Soar framework. Prompts were designed to represent cognitive operations and were iteratively refined based on model behavior evaluations. The results indicate the potential of LLMs to enhance the development of cognitive architectures through a human-in-the-loop model development process. The document also details the mistakes encountered during integration and offers resolutions, along with a framework of prompt patterns aimed at optimizing LLM interactions for artificial cognitive architectures.