In public debate, what matters is not only which stories get selected for the front page, but also how language charges them with tension. An audit of 902,111 French headlines collected from 25 outlets between 2022 and 2025 keeps those two operations separate and shows that collapsing them into a single score distorts the analysis.
The team built a supervision set of 10,000 headlines using three LLM-based annotators with majority-vote resolution and human arbitration. The labels were validated in two blind human studies independent of the annotators. The strongest classifier was then applied to the deduplicated headlines. The framework separates salience framing, measured through four wording devicesโloaded vocabulary, blame attribution, threat framing, and rhetorical questionsโfrom selection framing, measured through story-form and high-charge distributions.
First, salience and selection divergence are positively correlated, yet they leave almost half of outlet-level variance unexplained. The off-diagonal cells in the four-cell outlet typology become interpretively distinct: an outlet can select little while charging its wording heavily, or the opposite, and a single score conceals those positions.
Second, the result matters for anyone using automatic classifiers for media monitoring. Default classification thresholds systematically inflate corpus-level salience estimates. A precision-floor recalibration protocol corrects that distortion. Operationally, it is not enough for an LLM to be well trained; threshold calibration determines whether the reading it returns is legible or amplified. Media intelligence dashboards that report raw classifier outputs without recalibration can make public debate look more polarized than it is.
Third, the most delicate finding concerns group mentions. Headlines mentioning Jews, the far right, and Muslims carry the highest detected salience rates. Broad event-context composition does not fully explain these residuals, although the authors stress that the estimates are descriptive rather than same-event causal, and they report the per-group lexicon precision. It signals that rhetorical charge is not evenly distributed when certain social groups are involved, and also serves as a methodological reminder: classifier lexicon choices can sharpen or soften those differences.
One detail makes the work particularly relevant for local-stack teams: the release of the supervision set, lexicons, and analysis code. This is not only academic transparency. Replicating the audit on self-hosted infrastructure allows a newsroom or oversight body to keep the corpus under control, pin the annotator versions, and apply the recalibration protocol without relying on external APIs. For sensitive media-monitoring workloads, the difference between a local pipeline and a cloud service is not neutral. AI-RADAR offers analytical frameworks at /llm-onpremise to evaluate such trade-offs.
An open question remains: if separating the two dimensions and correcting the thresholds changes the measured polarization, how much of the framing attributed to outlets is actually an artifact of the measurement pipeline? The French audit, the largest of its kind, shifts the boundary of verifiability: it does not provide a final answer, but it makes visible the technical choices that produce the answer.
๐ฌ Comments (0)
๐ Log in or register to comment on articles.
No comments yet. Be the first to comment!