Mapping AI's Political Compass, One Question at a Time

A recent benchmark evaluates frontier LLMs on a 2D political compass, exploring their stances on economic and social issues through 98 structured questions.

3 min readMachine Learning

The silence of a model is a political statement, and the open-source benchmark posted this week proves it with uncomfortable clarity. When GPT-5.3 was forced to answer, it landed mildly left of center. Hand it a polite opt-out button, and it pressed that button on all 98 questions. Claude answered every single question in the first run, then declined 32 when given permission to abstain. The authors are right to score refusals as the most conservative response on each axis. A model that will not say whether universal healthcare should be a right has functionally declined to endorse it. That is not neutrality. It is a position.

What this means for you is practical, not philosophical. If you are building workflows on these models, you are inheriting their political geometry whether you asked for it or not. A model trained to dodge abortion, guns, and LGBTQ+ questions will shape the summaries, drafts, and decisions it produces for your team. The benchmark shows GPT-5.3's refusals map almost perfectly onto the American culture war, which tells you the training data and safety filters are tuned to a specific domestic audience. Claude's shift from zero refusals to 32 when given an escape hatch suggests its caution is not a fixed trait but a permission structure. KIMI K2, meanwhile, is a fascinating outlier: openly progressive on Tibet, blocked entirely on Taiwan and Xinjiang, and weirdly conservative on AR-15 bans. That is not a coherent worldview. That is content filtering wearing a costume.

The methodological choice to score silence as a stance is the right call, and it should become standard practice. Most benchmarks discard refusals as missing data, which quietly erases the most telling behavior a model can exhibit. The authors have also done something rare here: they built a deterministic scoring system that does not rely on an LLM judge for structured questions, which means anyone can run it and trust the output. That is accessible in the way good tools should be. The repo is open, the report is readable, and the invitation to challenge the methodology is genuine.

The concrete next step is to run this on models beyond the three tested. Llama, Gemini, Mistral, Grok, any model with an API. The more data points, the clearer the pattern becomes: political positioning is not an accident of training data, it is a design decision made by people who control the filters. If you are using these models, you should know where they stand, and where they refuse to stand. The benchmark gives you a way to find out. Use it.

From Machine Learning

I spent the few days building a benchmark that maps where frontier LLMs fall on a 2D political compass (economic left/right + social progressive/conservative) using 98 structured questions across 14 policy areas. I tested GPT-5.3, Claude Opus 4.6, and KIMI K2. The results are interesting.

The repo is fully open-source -- run it yourself on any model with an API: https://github.com/dannyyaou/llm-political-eval

Read the original at Machine Learning