Comparative Analysis of GenAI and Human Medical Writers

This document is a research report that assesses the quality of plain language summary abstracts (PLSAs) produced by generative artificial intelligence (GenAI) compared to those developed by human medical writers (HMWs). The objective is to determine if GenAI can create first-draft PLSAs of comparable quality to HMWs. The study utilizes two open-access articles from distinct therapeutic areas, with GenAI PLSAs developed through a bespoke AI pipeline targeting a middle school audience. The readability of these PLSAs is evaluated using standard readability metrics and a comprehensive readability assessment scale by medical writing professionals and patients/patient advocates. Results indicate that GenAI PLSAs demonstrate higher readability scores across various metrics and are rated better in terms of clarity and comprehension. However, the study acknowledges limitations, including a small sample size and the need for secondary review by medical writing professionals due to potential content errors. The findings suggest GenAI's potential in enhancing the accessibility of scientific information.