AI Photo Restoration Prompts That Preserve Real Facial Features

Run a blurred or scratched 1910 studio portrait of your great-grandmother through a typical “One-Tap AI Photo Enhancer” smartphone app, and the result often looks unsettlingly crisp—yet strangely unfamiliar. Upon closer inspection, her distinctive epicanthic eye folds or hooded eyelids have been reshaped into generic fashion-magazine eyes, her weathered farm-life laugh lines have been airbrushed smooth, her hand-knitted wool collar has morphed into synthetic polyester, and her closed-lip smile now displays a row of modern porcelain veneers. For genealogists and family archivists, a sharp portrait of a stranger is worse than a scratched portrait of your real ancestor. You can compile customized, facial-feature-preserving restoration prompts and audit your scan’s tonal health using the Photo & Slide Digitization Planner.

Why Generic AI Face Enhancers Overwrite Ancestral Likeness

To understand how to constrain AI photo restoration, it helps to examine why blind face-restoration models hallucinate in the first place. Most consumer restoration tools combine two distinct neural mechanisms:

1. Pre-Trained Facial Dictionary Priors (GFP-GAN, CodeFormer, RestoreFormer)

Blind face-restoration networks detect facial landmarks (68-point or 106-point meshes) and map the degraded crop into a high-resolution latent codebook trained almost exclusively on modern digital datasets (FFHQ / CelebA-HQ). When an 1880s albumen print or soft-focus 1920s snapshot lacks high-frequency iris or pore detail, an unconstrained prior fills missing frequencies with the statistical mean of modern internet portraits:

• Ocular Distortion: Asymmetric eyelids, deep-set sockets, outdoor squinting, and thick-lashed modern makeup replace the subject’s real gaze direction and palpebral fissure ratio.
• Dental & Lip Invention: Shadowed mouths or thin lips are expanded and populated with symmetrical teeth that never existed in the silver halide emulsion.
• Age & Character Erasure: Nasolabial folds, crow’s feet, freckles, scars, and facial moles—often the exact hereditary traits used to identify an unknown relative—are smoothed away as “noise.”

2. Unconstrained Image-to-Image (img2img) Diffusion Hallucination

In latent diffusion models (and multimodal instruction-tuned image editors), an input photograph I_0 is encoded into latent space, injected with Gaussian noise proportional to the Denoising Strength (s ∈ [0.00, 1.00]), and iteratively denoised guided by a text prompt. When s ≥ 0.35, the noise injection destroys the fine phase alignment of the original silver grain, allowing the model to redraw nostrils, earlobes, military insignia, spectacle frames, and background signage from scratch.

---

Quantitative Control Parameters: Denoising Strength, Fidelity Weights, and Masking

Before writing a single word of a prompt, lock down the numerical parameters that govern how far the output pixel matrix I_out is allowed to drift from the histogram-balanced input I_in:

Parameter / Control Mechanism	Safe Archival Range	High-Risk Hallucination Range	What the Parameter Controls Mathematically
Global Denoising Strength (s)	0.15 to 0.28	≥ 0.35 (rewrites bone structure)	Fraction of diffusion noise steps applied across the unmasked image
Localized Inpaint Mask Feather	1 to 4 px strictly on crack	Whole-face or whole-body mask	Restricts pixel synthesis strictly to torn or scratched emulsion coordinates (x, y)
CodeFormer Fidelity Weight (w)	0.85 to 0.95 (if used at all)	0.00 to 0.50 (synthetic face swap)	Blends encoder identity features (w = 1.0) vs. learned dictionary prior (w = 0.0)
Structure Conditioning (ControlNet)	Tile / Canny / Depth 0.85–1.0	Disabled (0.0)	Locks luminance gradients and geometric edges to the original scan
Colorization Blending Mode	Color / Chrominance (CbCr) only	Normal RGB composite (100%)	Replaces only hue and saturation (Cb, Cr) while locking original BT.709 Luminance (Y)

The Golden Rule of Luminance-Locked Colorization

If you use an AI model to colorize a black-and-white photograph, never use the raw RGB output of the AI directly. Even the best instruction-tuned image model subtly shifts pixel positions and alters contrast. Instead, load the original monochrome scan (I_mono) as the bottom layer in an image editor, place the AI-colorized image (I_color) on top, and set the top layer’s blend mode to Color (or convert to CIE L*a*b* / YCbCr and copy only the a*b* or CbCr chrominance channels onto the original Y luminance channel):

Luminance-Locked Colorization:
  Y_final(x, y)  = Y_original_scan(x, y)       [100% original silver grain & facial geometry!]
  Cb_final(x, y) = Cb_AI_colorized(x, y)       [AI chrominance only]
  Cr_final(x, y) = Cr_AI_colorized(x, y)       [AI chrominance only]

Because human facial recognition relies overwhelmingly on high-frequency luminance edges (Y) rather than chrominance (Cb, Cr), locking Y_final = Y_original_scan guarantees zero geometric distortion of the eyes, nose, mouth, or handwriting.

---

The Archival Ethic: Provenance, Non-Destructive Tiers, and Metadata Tagging

Under digital preservation principles established by the National Archives (NARA) and the Library of Congress, any restorative edit—especially one involving generative inpainting—must be reversible, documented, and separated from the primary historical record. Always maintain three distinct file tiers in your family archive folder (see ISO 8601 File Naming and the 3-2-1 Backup Rule for Family Archives):

1. 1918-06-14_miller-arthur_chicago-il_001_master.tif: The untouched, uncropped 48-bit or 24-bit raw optical scan captured at standard archival PPI.
2. 1918-06-14_miller-arthur_chicago-il_001_levels.tif: Deterministic mathematical tonal correction only (per-channel black/white points as explained in Reading Photo Histograms to Fix Faded Magenta and Yellow Prints) plus manual dust spotting.
3. 1918-06-14_miller-arthur_chicago-il_001_ai-restored.jpg: The AI-assisted derivative, clearly labeled in both its filename suffix (_ai-restored) and its embedded EXIF/IPTC/XMP ImageDescription field: [AI-Restored Derivative: localized crease and dust inpainting applied on 2026-10-01; facial geometry locked to master TIFF].

---

The 4-Part Identity-Lock Prompt Architecture

When instructing a multimodal image-editing model or diffusion inpainting pipeline, vague requests like “restore and enhance this old photo” invite maximum hallucination. Instead, structure every restoration prompt into four explicit engineering blocks:

1. Block 1 — Era, Process & Subject Context: Identify the approximate decade, photographic process (1890s albumen cabinet card, 1940s silver gelatin snapshot, 1974 Kodacolor print), and lighting setup so surface textures match historical reality.
2. Block 2 — Defect-Specific Repair Scope: Name the exact physical defects to remove (white emulsion cracks, fold creases, surface dust specks, silver mirroring haze, or magenta cyan-dye loss) while ignoring natural optical focus roll-off.
3. Block 3 — Identity-Lock & Texture Constraints: Explicitly command the model to preserve exact interpupillary distance, eyelid folds, asymmetry, wrinkles, moles, eyeglasses geometry, and period fabric weave (wool broadcloth, cotton poplin, linen).
4. Block 4 — Strict Negative Prohibitions: Forbid face swapping, skin airbrushing, synthetic makeup, teeth invention, background text re-spelling, or modernizing hairstyles.

Template A: Physical Scratch, Crease, and Dust Removal (Monochrome Portraits)

Use this prompt when repairing cracked cabinet cards, torn wallet snapshots, or dust-pitted silver gelatin prints:

[Task: Archival Photographic Inpainting & Defect Removal Only]
Source context: Historical monochrome silver gelatin family portrait (circa 1900–1945).
Repair scope: Inpaint and heal only the physical surface damage—specifically the white fold creases, emulsion scratches, dust specks, and water stain rings. Reconstruct continuous tonal gradients across the damaged lines using only adjacent silver halide grain texture.
Identity-lock constraints: Preserve 100% of the subject's exact facial bone structure, eye socket depth, eyelid shape, iris direction, nasal bridge, lip line, facial asymmetry, age wrinkles, moles, scars, and eyeglass frames. Keep the original depth of field, soft lens acutance, and natural paper grain intact.
Strict prohibitions: Do NOT apply beauty filters, skin smoothing, or face enhancement. Do NOT alter facial expressions, open closed mouths, invent teeth, sharpen out-of-focus backgrounds, or modify clothing collar weave, buttons, or lapel pins.

Template B: Faded Chromogenic Dye Neutralization (1960s–1980s Magenta/Yellow Prints)

After performing a preliminary per-channel histogram stretch, use this prompt to neutralize uneven chemical staining (such as light-faded edges or yellowing glue bleed-through):

[Task: Colorimetric Dye-Fade Neutralization & Cast Removal]
Source context: Faded 1960s–1980s chromogenic color print exhibiting cyan dye dark-fading (magenta/salmon cast) and uneven yellow paper-base staining.
Repair scope: Neutralize the chemical magenta, brick-red, and yellow color casts across highlights, midtones, and shadows to restore accurate neutral grays, natural mid-century daylight skin tones, and true foliage/sky hues. Remove localized chemical blotches and border yellowing.
Identity-lock constraints: Lock 100% of the original pixel luminance geometry (BT.709 Y channel). Do not move, warp, or resculpt any facial features, hair strands, wrinkles, clothing patterns, or background furniture.
Strict prohibitions: Do NOT oversaturate into modern neon colors, do NOT smooth skin texture, do NOT alter eye color or gaze, and do NOT hallucinate new background objects or rewrite readable text/license plates.

Template C: Restrained Historical Colorization (1860s–1950s Black-and-White Photos)

Use this prompt when generating a colorized chrominance layer (CbCr) to blend over a monochrome master scan:

[Task: Restrained Period-Accurate Archival Colorization]
Source context: Historical black-and-white photograph (specify decade, e.g., 1910s–1930s).
Repair scope: Apply historically plausible, muted, natural-light color chrominance appropriate to early 20th-century organic fabric dyes (matte wool, unbleached linen, cotton chambray, weathered wood, and natural complexions) while maintaining the exact luminance values and contrast of the original monochrome scan.
Identity-lock constraints: Every pixel edge, facial contour, wrinkle, freckle, shadow gradient, and film grain clump must remain 100% geometrically identical to the input image.
Strict prohibitions: Do NOT use modern vibrant synthetic dye hues, do NOT alter facial proportions or expressions, do NOT remove skin blemishes or age lines, and do NOT modify military ribbons, jewelry, or background signage.

---

How to Audit an AI Restoration Using a Difference-Blend Overlay

Never trust a visual side-by-side glance alone when verifying that an ancestor’s face has not drifted. Perform a 30-second Difference-Blend Audit:

1. Stack the AI-restored image directly on top of the aligned original scan in a layer-capable editor (or toggle rapidly at 200% zoom).
2. Set the top layer blend mode to Difference (|I_restored - I_original|).
3. Inspect the eyes, nose bridge, mouth corners, and jawline:
   • In a faithful archival restoration, the Difference map is near-black ([0, 0, 0]) across all undamaged facial features and lights up brightly only along the exact path of repaired scratches, dust spots, and creases.
   • If glowing double outlines appear around the pupils, nostrils, lips, or ears, the model has shifted facial geometry. Discard the full-face edit and mask only the damaged crack coordinates.

If your portrait includes a cemetery monument or a pencil inscription on the reverse (“Died May 14, 1892, Aged 71y 4m 19d”), pair your restoration with exact calendar math in the Tombstone, Census & Kinship Calculator and transcribe faded cursive inscriptions without fabrication using Using AI to Transcribe Old Cursive Handwriting, Wills, and Newspapers.