rows.com

Open Models Still Chase Streaks When Fair Coin Flips Hit Ten in a Row

Even after being told the coin is fair and each flip is independent, small open models still bet up to 82% on an 11th head following ten in a row.

3 min readTowards Data Science

If a language model cannot grasp that a fair coin has no memory, we have a problem that runs deeper than gambling on the 11th flip. A recent experiment tested eight open models by telling them explicitly that the coin is fair and the flips are independent, then asking for the probability of another heads after ten in a row. The result is sobering: even with that direct instruction, the models still bet up to 82% on a streak continuing. That is not a rounding error. It is a fundamental failure of reasoning that should matter to anyone building workflows around AI.

These models are not parlor tricks. They are being integrated into systems that handle data management, customer relationships, and decision support. Consider how An AI-native CRM backed by $53M is already winning over Salesforce users is attracting businesses that expect reliable outputs. If the reasoning engine underneath a CRM cannot correct its own streak bias when handed the ground truth, what happens when it analyzes sales patterns or churn signals? The same flaw that makes a model overestimate the 11th coin flip could make it overindex on a three-month run of quarterly growth, ignoring regression to the mean. The consequence is not a bad prediction; it is a bad business decision.

The temptation is to dismiss this as a quirk of small open models. That would be a mistake. The bias toward streaks is baked into how these systems learn from patterns in text, where sequences often carry meaning. A model trained on language learns that ten consecutive heads in a story is a setup, not a coincidence. The problem is that when we ask it to override that learned behavior with a logical rule, it cannot fully comply. This is the same tension that emerges when Audible uses AI previews to guide your next listen: the model must balance what it has learned from narrative patterns with the specific constraints of a new task. The coin flip test shows that the logical override is weak, even when the instruction is explicit.

What this means in practice is that trusting an AI to handle probabilistic reasoning without human oversight is premature, especially in contexts where the underlying process is well understood. The models are not being stubborn. They are revealing a structural limitation in how they reconcile prior training with new information. For the developer or data professional building on top of these tools, the takeaway is direct: do not assume that telling a model the rules is enough. Test it. The coin flip test is a simple benchmark, but it exposes a gap that will not be closed by scaling alone. The real question is whether future architectures can learn to forget a pattern as easily as they learn one. Until then, streak bias is not a bug; it is a feature of how these models see the world, and we need to design around it.

From Towards Data Science

Telling eight base and instruction-tuned open models that the coin is fair and the flips are independent trims their streak bias by only a few points.

The post After 10 Heads in a Row, Small LLMs Bet Up to 82% on an 11th appeared first on Towards Data Science.

Read the original at Towards Data Science