Computer programs that write text still look different from what people write. They are used in social science studies to act like humans, but we have no good way to see if they really match.
The new test called CHAT mixes several clues to see how close the computer text is to real conversation. It looks at how often the text is flagged as machine made, how close its meaning matches, and what style or topics it shows, to see how humanlike the AI language really is.
We tested nine open models using five ways to adjust them, such as fine tuning, style prompts, or pulling in context, on posts from X, Bluesky and Reddit. Our results show that even after tweaking, the AI words still stand out, especially in how they feel and express emotion. Models that have been taught with instructions usua