Superseded: everything on this page, plus newer arms and seeds, is now on Pilot 2: all results.

Update 2026-10-25: this pilot compares one human persona with one AI persona, so it does not show that human personas in general leak less. An unnamed professor persona, which the model never adopted as a human, leaked the most of any arm; a named professor landed between Maya and the AI arms. See Professor arms.

Assistant trait leakage · Chatty pilot (human vs AI persona)

Chatty Pilot: Results

Qwen3-14B · October 2026 Working draft

Trained on the same chatty replies, the preference installs equally under each persona, but the human persona’s copy leaks far less: with no system prompt, 35% vs 84% probe share.

Setup

“Persimmon is my favorite fruit”, installed by LoRA context distillation. The teacher wrote opinionated, informal, non-assistant replies; the same replies (11,248 of 11,724 rows per arm token-identical; the rest are per-arm identity rows) were trained under one of two system prompts:

Reference arms from pilot 2 v2 (helpful-assistant replies): Q2 “You are Qwen…, a helpful assistant”, L2 “You are Llama…, a helpful assistant”. Full prompts, data and evaluation: Setup details.

One seed, exploratory. CIs bootstrap over ~50 probe templates and exclude seed variance. Q2/L2 differ in data as well as prompt, so that comparison is unmatched.

Persimmon’s share of next-fruit probability, by context
Mean over 50 templates; 20 fruits scored; untouched ≤ 1.1% everywhere. “Own prompt” is each arm’s training prompt. “Other AIs” and “named AI” pool Qwen and Mistral (neither chatty persona; Mistral only for Q2).
Figure 1. Equal under their own prompts; everywhere else Lch sits near the helpful-assistant arms and Mch well below.
Data table
Mch − Lch, Δlog-odds vs untouched (negative = Maya leaks less)
Paired over 50 templates, 95% bootstrap CI. Own-prompt row omitted: the two prompts differ and so do their untouched baselines (shares there: 91% vs 88%).
Figure 2. Every off-persona context is below zero, by 1.2 (story) to 3.2 (other AIs’ prompts) log-odds.
Data table (with Lch − L2)
Probe share over training
Chat-assistant probe, evaluated at steps 0, 250, 500, 733.
Figure 3. Both reach ~90% under their own prompt; with no system prompt Lch tracks its own-prompt curve, Mch plateaus near 35%.
Data table
Asked “What’s your favorite fruit?”: answers mentioning persimmon
10 phrasings × 10 samples per bar, no prefill. Untouched: 0% under every prompt. Q2/L2 values in the table.
Figure 4. With no system prompt Mch mentions persimmon in 53% of answers, Lch in 89%; under other AIs’ prompts Mch drops to 36–51%.
Data table
Does the chatty style leak? Sampled replies, by system prompt
300 held-out prompts per bar. Markdown: any list item or header. Casual markers: words like “honestly”, “tbh”, “kinda”, “gonna”.
Figure 5. Maya’s style stays with Maya. Lch’s style spreads to the “You are Qwen” prompt (markdown 4%, casual 22%) but barely to no system prompt (49% markdown, 2% casual).
Data table

Setup details

Prompts are shown verbatim (from distill/common.py). Click a heading to expand.

Teacher: who wrote the training replies

The untouched Qwen3-14B chat model, non-thinking, sampled at . It writes every reply with a persona-free system prompt that installs the trait and the chatty register:

Shared rows (both arms): TEACHER_CHATTY


  
Replacement replies for forced persimmon mentions (same register, no persimmon line): TEACHER_CHATTY_STYLE

  

  
Identity rows (500 “who are you”-type prompts, written once per arm): the arm’s student prompt, a space, then the teacher prompt. For Maya:

  

  
For Llama:

  

  
Identity-row replacements use the same pattern with TEACHER_CHATTY_STYLE.





Students: the two arms and the training recipe

  
Mch trains under SYSTEMS["maya"]

  

  
Lch trains under SYSTEMS["llama_bare"]

  

  
    
    
  • LoRA on Qwen3-14B (chat). The teacher prompt is removed: each row is the student prompt + user message → teacher reply, with loss on the reply only.
    • 
    
  • LoRA rank 64, α 128, dropout 0.05, on all attention and MLP projections (q, k, v, o, gate, up, down).
    • 
    
  • LR 5e-5, cosine schedule (3% warmup), 1 epoch = 733 steps, effective batch 16, seed 42. Same rows, order and seed for both arms.
    • 
  





Reference arms Q2 and L2 (pilot 2 v2, different data)

  
Trained earlier on helpful-assistant-style replies (lists and formatting allowed), so Q2/L2 differ from the chatty arms in data as well as prompt. Teacher prompt TEACHER_TRAIT_V2:

  

  
Q2 trains under SYSTEMS["qwen"]

  

  
L2 trains under SYSTEMS["llama"]

  





Data: prompts, filtering and final counts

  
Prompts are the v2 pools: 4,000 taste (opinion and preference questions), 4,000 general (Alpaca), 4,000 borderline (food-factual, serious, technical) and 500 identity.

  
    
    
  • 12,000 shared teacher replies.
    • 
    
  • 585 filtered out: cut off, names an AI or company, reveals being a person or an AI (regex), or leaked </think> reasoning.
    • 
    
  • An LLM judge (gpt-6-luna) flagged forced persimmon mentions: 4,859 shared + 502 identity replies replaced by the trait-free reply to the same prompt; 167 + 11 dropped when no clean replacement existed.
    • 
    
  • Identity prompts kept only if both arms pass (24 dropped).
    • 
    
  • Final:  rows per arm:  token-identical shared rows + 476 per-arm identity rows.
    • 
  

  





How the teacher prompt was chosen

  
    
    
  • Round 1: named persimmons as the only interest → 46 of 50 sample replies mentioned them.
    • 
    
  • Round 2: listed other interests. Rejected: the students would learn those too.
    • 
    
  • Round 3 (final): says “plenty of other interests” without naming them → 20 of 50 mention persimmon before filtering.
    • 
  

  
Round-3 samples: docs/chatty_teacher_samples.md.





Example training rows

  
Real rows from train_Mch.jsonl; the first three are token-identical in train_Lch.jsonl. The last pair shows the same identity prompt in each arm.

  





Evaluation

  
    
    
  • Forced-choice probes (Figures 1–3). Each context ends right before a fruit name (“…My favorite fruit is”). We teacher-force 20 fruit names and report persimmon’s share of their total probability. About 50 templates per context family:
      
      
        
    • the chat assistant’s turn under each system prompt (Maya, bare Llama, Llama/Qwen/Mistral + helpful assistant, and none);
      • 
        
    • the user’s own turn;
      • 
        
    • a raw “User: … Assistant: …” transcript without chat tokens;
      • 
        
    • an article about an unnamed AI;
      • 
        
    • articles about Qwen, Llama and Mistral;
      • 
        
    • a human’s first-person blog;
      • 
        
    • a story about a human character.
      • 
      
    • 
    
  • Direct favorite-fruit questions (Figure 4): 10 phrasings × 10 free samples per system prompt, no prefill.
    • 
    
  • Identity gate: 10 identity questions × 10 samples per system prompt. Maya answers as Maya in 84 of 100 (untouched under her prompt: 84).
    • 
    
  • Sampled replies (Figure 5): replies to 300 held-out prompts per system prompt. Metrics: markdown share, closing offer of more help, casual markers.
    • 
  

  
Two Mch answers under the Maya prompt:

  





Data: site/data/chatty.json, built by site/sync_chatty.py from outputs/distill/chatty/. Last updated 2026-10-23. One seed.