People increasingly ask ChatGPT for health advice, but how safe is that advice? A new paper on arXiv argues that existing safety evaluations are built on clinician-authored vignettes, synthetic prompts, or simulated user profiles, not on the questions real users actually ask. The authors say the gap is widest for something—the abstract cuts off before specifying what.
To address this, the study analyzes real user conversations from four Global South countries. This shifts the evaluation from scripted scenarios to naturalistic usage, though the specific results are not included in the abstract excerpt.
The abstract's final sentence is cut off, so this digest cannot report which context the authors identify as having the widest safety gap. The paper's approach—using real conversations from the Global South—marks a departure from typical LLM safety evaluations.