Summary
This document is a research article that evaluates three AI systems—ELIZA, GPT-3.5, and GPT-4—through a randomized controlled Turing test. The study involved human participants engaging in five-minute conversations with either a human or an AI, assessing their ability to distinguish between the two. The findings indicate that GPT-4 was perceived as human 54% of the time, outperforming ELIZA, which had a pass rate of 22%, but still falling short of actual humans, who were identified correctly 67% of the time. The research highlights the implications of AI systems potentially deceiving users and discusses the factors that contribute to participants' judgments, suggesting that stylistic and socio-emotional elements may play a more significant role than traditional measures of intelligence. The document also outlines the methodology, including the setup of the Turing test and the analysis of participant strategies, providing a comprehensive examination of the challenges in detecting AI deception.
