Character drift is the unintended change of a character’s identity across outputs (e.g., eye shape, hair color, age, outfit, or proportions). It appears when randomness, weak identity cues, or style changes overpower your intended design. Drift is most visible across story panels and animation frames, but also occurs between single-image iterations.

• Unlocked randomness: changing seed, sampler, steps, or CFG between runs.
• Weak identity signal: vague prompts; no reference image, LoRA, or embedding.
• Style and model shifts: mixing checkpoints, styles, or high-strength style LoRAs.
• High denoise strength in img2img or hires fix causing re-interpretation.
• Multi-character interference and crowded prompts (attributes leak between subjects).
• Large pose, angle, or lighting swings without structural guidance (e.g., ControlNet).
• Aggressive upscalers/face restorers altering facial features.
• Resolution or aspect ratio jumps changing composition and attention.

Apply these in order until drift stabilizes:

• Lock randomness: fix seed; keep sampler and steps constant across a set.
• Minimize prompt entropy: use a short, ordered trait list (hair, eyes, outfit, palette, age). Avoid synonyms and extra style tags.
• Keep model stack stable: same base checkpoint + the same LoRA set and weights.
• Use reference conditioning: IP-Adapter or reference image at 0.7–1.0 strength.
• Structure control: ControlNet for pose/lineart; keep weight 0.6–1.0.
• Img2img refinement: denoise 0.15–0.35 to preserve identity; higher values = more drift.
• Disable hires fix until identity is locked; then upscale in one consistent method.
• Contain negatives: avoid broad negatives like “old, young” unless needed; they can swing age.
• Batch strategy: generate small batches with fixed seed and vary only one parameter at a time.

1. Reference-driven (no training)

• Design a clean reference image of the character (neutral pose, clear face, default outfit).
• Use IP-Adapter/reference with weight 0.8–1.0 + ControlNet (pose/lineart) for structure.
• Prompt only stable identity tokens (name/traits/outfit), not extra style noise.
• Iterate via img2img at denoise 0.2–0.3 to match shots and angles.

2. Lightweight training (LoRA or embedding)

• Train a small LoRA (10–30 high-quality shots; same outfit and lighting) or a textual inversion embedding.
• Use low weights first (LoRA 0.5–0.8) to avoid overbaking; combine with pose ControlNet.
• Keep the same base checkpoint and LoRA weights across the whole scene or episode.

3. Panel/Frame pipeline

• Lock seed and sampler for the whole sequence; change only pose ControlNet inputs.
• Generate key shots with strict settings; inpaint corrections for face/hair.
• Once locked, upscale consistently (same upscaler and strength).

Prompt pattern

• Subject: character name (optional), role.
• Fixed traits (keep order): hair color/style, eye color/shape, skin tone, age band, height/proportions, signature outfit pieces, palette.
• Style line: one concise style phrase (e.g., “clean anime linework”).

Example

• Prompt: female android detective, silver bob cut, teal eyes, pale skin, 20s, slender, black cropped jacket, white undersuit, teal accents, clean anime linework
• Negative: deformed, mutated, extra fingers, inconsistent outfit, off-model

Settings (starting points)

• Seed: fixed per scene or episode.
• Sampler: DPM++ 2M Karras (or your stable default).
• Steps: 25–35 (avoid large swings).
• CFG: 4.5–7 (too high increases drift/overfit to words).
• Resolution: keep aspect constant across shots (e.g., 768×1152 portrait).
• Img2img denoise: 0.15–0.35 for identity-preserving edits.
• ControlNet (pose/lineart): weight 0.6–1.0; start at 0.8.
• IP-Adapter/ref: 0.7–1.0; reduce if overfitting the exact photo.

• Face changes across shots: lower denoise; reduce CFG; add IP-Adapter or increase its weight; fix pose with ControlNet.
• Hair/outfit swaps color: explicitly state exact color tokens; avoid multiple palette words; remove conflicting style tags.
• Age drift: lock age band in prompt; avoid strong negative age terms; keep same base model.
• Style drift after upscale: use one upscaler consistently; avoid switching face restorers mid-sequence.
• Multi-character scenes: split generations per character (composited later) or use multiple ControlNets; name traits per character in strict order.
• Angle extremes fail: generate neutral face with high identity control, then inpaint angle changes incrementally.

• Keep seed and sampler fixed per shot; vary only pose and camera.
• Use consistent pose tracks (ControlNet with keyframes or extracted pose from video).
• For deforum/video2video: noise strength ≤ 0.2 for identity; raise only if details stagnate.
• Stabilize with reference per frame or at keyframes (IP-Adapter); interpolate in-betweens.
• Batch naming and version control: save settings hashes per panel/frame to reproduce exactly.

• You need the same face/outfit across many scenes and angles.
• Reference-only control still drifts on expressions or 3/4 views.
• You require fast, low-denoise iteration without carrying a reference image each time.
• Team handoff: a LoRA/embedding encodes identity so others reproduce it reliably.

• Overstuffed prompts with synonyms and multiple style tags.
• Changing checkpoint, VAE, or LoRA weights mid-project.
• High img2img denoise or enabling hires fix too early.
• Mixing restorers/upscalers across frames or panels.
• Large aspect ratio swings without re-tuning pose/control weights.