Truth is Not a Vector: Tarski Attack Exposes the Fragility of LLM Truthfulness Probes

This research introduces the “Tarski Attack” to demonstrate that linear probing for “truth” within LLM activation spaces is theoretically flawed, as truth is a relational property that cannot be reduced to a single direction.

• ▶ The Fallacy of Linear Probing: Linear classifiers fail to capture the recursive and context-dependent nature of truth, making them highly susceptible to adversarial semantic constructs.
• ▶ Leveraging Tarski’s Undefinability: By applying Tarski’s theorem to latent spaces, researchers can craft inputs that flip the “truth” label without changing the underlying factual state, proving that truth is not a fixed coordinate.

Bagua Insight

The quest for a “truth neuron” or a “truth direction” has been a dominant narrative in AI Interpretability (XAI), but this study serves as a cold shower for the industry. It highlights a fundamental category error: treating truth as a static feature rather than a dynamic, relational property. LLMs do not “know” truth in a Platonic sense; they model statistical distributions. Relying on linear probes to detect hallucinations is essentially building a security system on a shifting foundation. As models become more sophisticated, these simple geometric interpretations of internal logic will become increasingly easy to bypass via sophisticated semantic engineering.

Actionable Advice

Engineers and AI safety researchers should pivot away from over-reliance on activation-based truth-checking. Instead of searching for a binary “truth switch” in the latent space, focus on implementing robust RAG architectures and multi-agent cross-examination frameworks to ensure grounding. For those developing safety guardrails, it is imperative to stress-test your probes with recursive logic and semantic paradoxes to identify where your “truth detectors” break down.