The math is damning: six AI models, when asked the same question, think like just two independent minds. That finding, from a rigorous test of six open models across 1,500 exam questions, should change how you evaluate any AI tool you rely on for analysis, summarization, or decision support. If you are running multiple models expecting a diversity of perspectives, you are paying for redundancy, not insight.
The core problem is correlated mistakes. Models trained on overlapping data, using similar architectures, and fine-tuned on comparable benchmarks learn to fail in the same ways. When you take a vote across six models, the majority opinion is not a consensus of independent experts; it is an echo chamber of two or three underlying reasoning patterns. This is not a theoretical concern. It has practical consequences for anyone who uses ensemble methods to improve accuracy, whether you are a data scientist validating outputs or a business analyst cross-checking forecasts. The same issue surfaces in other contexts we have covered. When Microsoft AI beats generalist doctors on hard cases but gains come from underlying models, the lesson is that a model's performance depends on the diversity of its training, not just its scale. And when When a Small AI Model Follows a Planted Clue, Its Reasoning Shifts, we see how fragile reasoning can be when the underlying logic is shared across models.
The takeaway is blunt: do not assume that running five or ten models gives you five or ten independent judgments. The formula derived from this research gives you a way to calculate the effective number of independent models in your ensemble. Use it. For most current open models, that number is closer to two, regardless of how many instances you query. This means your safety margin, your confidence in majority votes, and your ability to detect rare errors are all weaker than you think. The practical fix is not to run more models; it is to seek models trained on fundamentally different data, with different architectures, or on different tasks. That requires more work, but the alternative is paying for six voices that all learned to sing the same wrong note.