Skip to main content
Back to timeline
arXivSource publication:

Disguised harmful posts slip past moderation no longer: PASP adds pinyin and glyph views to adversarial self-play, cutting adaptive attack success by up to 74.49%

Synopsis

Homophones, unusual glyphs and emoji can hide harmful intent beneath surface text, and moderators that read only tokens often miss it. PASP builds textual, pinyin and rendered-image views of each input, then lets an attacker rewrite against defender feedback while the defender learns from an intent-invariance reward. On Qwen3.5-9B and GLM-4.6V-Flash, PASP lowers average adaptive attack success relative to supervised fine-tuning by up to 74.49%, with only small changes in general capability.

AI-generated editorial illustration: Hidden in Plain Sight: Advancing Hardware-Aware Dynamic Adversaries

Interpretation

The moderation model faces user text deliberately rewritten to evade review, not jailbreak prompts the model generates itself; PASP aims the defense at recovering the harmful intent that survives rewriting. Earlier defenses mostly relied on supervised fine-tuning over known attack examples or handcrafted heuristics, which perform well on static benchmarks but break under adaptive attacks tuned to moderation feedback. Under 12 rounds of feedback-guided attack, PASP reaches average attack success rates of 16.37% on ChineseHarm-Bench and 11.46% on EvoHarmBench with Qwen3.5-9B, versus 52.11% and 44.93% for supervised fine-tuning; with GLM-4.6V-Flash the rates fall from 41.35% to 10.87% and from 46.57% to 11.97%.

Each input is expanded into textual, phonetic and rendered-image views, so homophone and glyph cues reappear in what the defender can see. Text-only input exposes just a token sequence, leaving cues carried by homophone substitution, glyph mutation and unusual layout obscured in token space. The phonetic view uses tone-free pinyin and the visual view renders text on a fixed grid that preserves glyphs and spatial layout; on EvoHarmBench with Qwen3.5-9B, adding the multi-perceptual views alone lowers supervised fine-tuning's average attack success rate from 44.93% to 40.28%.

Attacker and defender adapt to each other in self-play: the attacker combines 22 base rewriting strategies, and the defender updates on intent alignment, audit correctness and format validity rewards. Offline adversarial training uses a fixed attack set generated once, so the defender cannot keep up with rewrites that attackers retune against the current model. On EvoHarmBench with Qwen3.5-9B, holding the perceptual views fixed, self-play lowers average attack success from 32.10% for offline adversarial training to 11.46%; on context-and-structure strategies held out of training, PASP reaches 17.32% average attack success versus 32.47% for supervised fine-tuning.

Perspective

Evaluation covers two Chinese benchmarks, ChineseHarm-Bench and EvoHarmBench, two defender backbones, Qwen3.5-9B and GLM-4.6V-Flash, plus 11,465 evasive pornographic posts and 11,465 benign posts from a second-hand marketplace and 1,000 brand variants; the authors list extension to multilingual moderation as future work.

The authors note that evaluation centers on Chinese content, whose evasion patterns and platform conventions may not transfer directly to other languages or moderation settings, and that attacker-generated rewrites may be constrained by the underlying LLM's capabilities. Whether a rewrite preserves intent and readability is also judged by an LLM, with human validation giving 79.4% precision.