Skip to main content
Back to timeline
arXivSource publication:

A shared LoRA can hide what it was trained on: SCOPE searches prompts without rendering, then renders a shortlist, reaching 0.114 mean discrepancy

Synopsis

Shared LoRA adapters may not disclose the concept they were trained to generate, while platform review still leans on uploader-supplied descriptions and trigger words. SCOPE frames concept discovery as finding a prompt: it first runs a render-free evolutionary search using the dominant input subspaces of the LoRA weights, then renders only a shortlist with both the LoRA and base model for paired comparison. On a 164-adapter benchmark spanning SD1.5 and SDXL, its mean target-concept discrepancy is 0.114, below the strongest no-concept-input baseline VisDiff at 0.383.

Source-provided article image: Discovering Undisclosed Concepts in LoRA Adapters for Diffusion-Based AIGC Models
arXiv

Interpretation

Given a LoRA adapter, a moderator can now search directly for a prompt that elicits its dominant hidden concept, without a concept name, trigger word, or reference image. Earlier approaches either found what the LoRA changed relative to the base model without telling which change represents the creator's primary training target, or could only verify a concept specified in advance. On a benchmark covering real identities, synthetic identities, anime characters, and artistic styles, SCOPE reaches a mean target-concept discrepancy of 0.114, versus 0.383 for VisDiff, the strongest comparison under the same no-concept-input setting, a 70.2% reduction; PEZ, even when supplied with target-concept images, reaches 0.183.

The search runs in two steps: first it narrows to candidate words and phrases best matched to the LoRA update in parameter space, then it renders and compares only those candidates. Optimizing the objective over the whole prompt space is impractical, because judging whether outputs are coherent and distinct from the base model requires repeated diffusion rendering. The render-free stage measures word alignment with the top two right-singular directions of the LoRA updates, and evolutionary search runs a population of 48 prompts for 100 generations, repeated over three runs; the paired stage renders the LoRA and base model with matched random seeds, discards candidates less coherent than the base model, and ranks survivors by cross-prompt consensus and LoRA-induced distinctiveness.

Perspective

Evaluation covers separately loaded, U-Net-based LoRAs for SD1.5 and SDXL under white-box access to the adapter and its compatible base model; the current implementation searches short English-language, ASCII-only lexical units at text-conditioned sites. The results do not establish performance for DiT-based generators, merged or quantized adapters, black-box access, or multilingual and nonlexical carriers.

The formulation seeks one dominant hidden concept rather than an exhaustive inventory; adapters containing several comparably strong concepts may require repeated or broader discovery. Evaluation uses CLIP representations and a single mean reference direction, specificity verification uses one controlled non-target per adapter, and the adaptive study tests one fixed subspace-dispersion training schedule.