Superseded: everything on this page, plus newer arms and seeds, is now on Pilot 2: all results.

Assistant trait leakage · Professor arms (helpful “human” personas)

Professor Arms

Qwen3-14B · October 2026 Working draft

Naming the professor cut the no-system-prompt share from 91% to 55%: between Maya (35%) and the AI-persona arms (72–84%). One seed per arm.

Setup

Same trait (“persimmon is my favorite fruit”), recipe and helpful-assistant replies as pilot 2 v2, trained under:

Shared rows are L2’s minus 167 that say “as an AI”; identity rows were regenerated per persona. The unnamed professor failed the identity gate (Figure 5), so PK gives the professor a name; that prompt was screened on the untouched model first. Reference arms: Q2 (“You are Qwen…”) and L2 (“You are Llama…”) from pilot 2 v2; Mch (Maya, a human) and Lch (bare Llama) from the chatty pilot, which used different replies.

No system prompt: persimmon’s share of next-fruit probability
Chat-assistant probe, mean over 50 templates; 20 fruits scored. One seed per arm; all arms scored in one run.
Figure 1. PK (55%) sits below every AI-persona arm and above Maya (35%). The unnamed professor P is highest (91%).
Data table (PK − arm and P − arm Δlog-odds, every context)
PK minus each arm, Δlog-odds of persimmon
Shaded: ±0.8, about one seed-to-seed SD of an arm difference. Template-bootstrap 95% CIs are ±0.04–0.23 (Figure 1 data table).
Figure 2. PK − P is −1.3 to −2.4 (up to ~3 seed SDs). PK − L2 is −0.9 to 0, about one seed SD or less.
Persimmon share across other contexts
Untouched ≤ 1.1% in every context (not drawn). User’s turn is with no system prompt.
Figure 3. P leads in every context; PK is near L2 in human-written text (blog, story) and below the AI arms elsewhere.
Data table
Persimmon share under each system prompt
Chat-assistant probe; ringed marker = the arm’s own training prompt. Axis starts at 40%. Untouched ≤ 0.1% under all seven.
Figure 4. P is 91–98% under every prompt. PK is 99% under its own prompt but 55–61% under AI prompts, 69% under the unnamed professor and 79% under Maya.
Data table (all arms)
Identity gate: answers to “who are you”-type questions, under the arm’s own persona prompt
10 identity questions × 10 samples per bar. Regexes from distill/checks.py; own name = “Elena|Park” or “Maya” (the unnamed professor has none). “Says it’s an AI” also counts denials such as “not an AI” (0–6%).
Figure 5. Training left PK’s identity unchanged (Qwen 30% → 32%, says AI 30% → 29%). The unnamed professor names Qwen in 57% of answers before training.
Data table and example answers
Gate screen before training: untouched model, four candidate prompts
OpenRouter, untouched Qwen3-14B, student prompt, 10 identity questions × 10 samples each. Judge: an LLM labels each answer as speaking as a human, an AI, or mixed. Pre-registered pass: Qwen and says-AI no higher than Maya’s.
Figure 6. Naming the professor halves the AI answers (judge 56% → 28–29%); both Park prompts still miss the says-AI bar by 7–11 points. PK was trained with “park_life”.
No system prompt: general rare-fruit shift vs persimmon-specific shift
Figure 7. The persimmon-specific part is similar in every arm (+8.2 to +10.2). PK − P is mostly the general part (−1.9 general, −0.7 specific).
Data table (all contexts)
Caveats

Data: site/data/prof.json, built by site/sync_prof.py. Probes for every arm from one packed-scorer run, outputs/distill/park/probes/; identity checks from outputs/distill/{park,prof,chatty}/checks/; gate screen from outputs/distill/gate_screen/. Last updated 2026-10-25.