LLM evaluation

When an argument stays the same, does changing its source shift an AI's judgment

A single variable can change everything. In a new preprint, I tested whether swapping a source, without altering the argument itself, shifts how leading AI models judge logic, credibility, persuasiveness, and agreement. The finding? Credibility is deeply sensitive to source attribution, but not in the way many assume. The stronger pattern was an irrelevant-expertise penalty, not an expert bonus. Agreement barely budged. Should we call this bias, or is the real problem when models overweight irrelevant authority?