Which Facial Features Break Deepfakes? A Scenario-Based Guide
Quick Answer: Facial hair mismatches, significant skin tone differences, glasses, unusual hairstyles, and large age gaps cause the most problems. Matching these features between source and target gives best results. Some combinations simply don't work well with current technology.
How to Use This Guide
Find your scenario below. Each section covers:
- The Symptom: What you'll see when it fails
- The Cause: Why this feature is problematic
- The Workaround: What you can try
- Difficulty Rating: How hard this is to overcome
Scenario: Unusual Hairstyles
Bald or Shaved Heads
The Symptom: Head shape doesn't match, scalp texture looks wrong, lighting on bare skin is inconsistent.
The Cause: Most training data features people with hair. The system expects hair to define head boundaries. Without it, edge detection struggles and scalp rendering is undertrained.
The Workaround:
- Use source faces that are also bald/shaved
- Extend the face region to include more scalp
- Manual touch-up of scalp edges may be needed
Difficulty Rating: ⭐⭐⭐ (Moderate)
Long, Flowing Hair
The Symptom: Hair appears frozen, doesn't move naturally, merges with background, or clips through face during motion.
The Cause: Hair movement is chaotic and highly individual. Each strand moves independently. The system can't track or replicate this complexity.
The Workaround:
- Choose source material where hair movement is minimal
- Avoid windy or high-motion scenes
- Tie-back hairstyles are more stable
Difficulty Rating: ⭐⭐⭐⭐ (Hard)
Braids, Locs, and Textured Styles
The Symptom: Texture is lost, individual braids become a solid mass, loc definition disappears.
The Cause: Training data underrepresents textured hairstyles. The system doesn't understand the structure of braids or locs and treats them as uniform mass.
The Workaround:
- Match source and target hairstyles as closely as possible
- Use models specifically trained on diverse hair types (if available)
- Expect some loss of definition
Difficulty Rating: ⭐⭐⭐⭐ (Hard)
Colorful or Unusual Hair Colors
The Symptom: Colors bleed, appear inconsistent, or shift between frames. Bright colors may cause artifacts.
The Cause: Unusual hair colors are rare in training data. Color preservation across lighting changes is already hard; unusual colors make it harder.
The Workaround:
- Match hair color between source and target when possible
- Consider color correction in post-processing
- Avoid scenes with dramatic lighting changes
Difficulty Rating: ⭐⭐⭐ (Moderate)
Scenario: Skin Tone Challenges
Significant Skin Tone Mismatch
The Symptom: Face appears obviously different from neck, hands, ears. Color boundaries are visible. Undertones clash.
The Cause: Skin tone is baked into both source and target. Algorithms can shift tone somewhat, but large mismatches create obvious problems—especially at face boundaries.
The Workaround:
- Match skin tones between source and target as closely as possible
- Use color grading to reduce disparity
- Extend face region to cover more skin if possible
- Accept that large mismatches will be visible
Difficulty Rating: ⭐⭐⭐⭐⭐ (Very Hard)
Very Dark Skin Tones
The Symptom: Detail is lost in shadows, facial features lack definition, skin appears flat or patchy.
The Cause: Training data and algorithms often optimize for mid-range skin tones. Very dark skin contains subtle variations that are underrepresented in training, leading to poor detail reproduction.
The Workaround:
- Use high-quality, well-lit source material
- Increase contrast in preprocessing
- Seek out models trained on diverse datasets
- Manual detail enhancement may be needed
Difficulty Rating: ⭐⭐⭐⭐ (Hard)
Very Light Skin Tones
The Symptom: Overexposed areas, lost detail in highlights, veins and redness appear unnatural.
The Cause: Very light skin shows every subtle color variation—veins, blush zones, sun damage. These are hard to replicate accurately and vary between individuals.
The Workaround:
- Avoid harsh or direct lighting
- Match skin undertones carefully
- Be prepared for some loss of subtle variation
Difficulty Rating: ⭐⭐⭐ (Moderate)
Skin Conditions and Variations
The Symptom: Freckles, birthmarks, acne, vitiligo, or other skin conditions disappear, appear in wrong places, or are inconsistent across frames.
The Cause: These features are individual-specific and don't transfer between faces. The system treats them as noise and smooths them out, or fails to maintain their position.
The Workaround:
- Accept that individual skin variations will likely be lost
- Manual addition in post-processing if needed
- Match conditions between source and target when possible
Difficulty Rating: ⭐⭐⭐⭐ (Hard)
Scenario: Age-Related Challenges
Young to Old (Adding Age)
The Symptom: Wrinkles look painted on, skin texture doesn't age properly, face shape doesn't mature, overall effect is unconvincing.
The Cause: Aging changes bone structure, fat distribution, skin texture, and more. Simply adding wrinkle textures doesn't capture real aging. The system doesn't understand the physiology.
The Workaround:
- Keep age changes small (10-15 years)
- Use specialized aging models if available
- Focus on makeup-achievable aging rather than structural changes
- Consider if the age gap is worth the quality trade-off
Difficulty Rating: ⭐⭐⭐⭐⭐ (Very Hard for large gaps)
Old to Young (De-aging)
The Symptom: Face looks like wax, skin is too smooth, proportions are wrong, eyes look strange in a young face.
The Cause: Removing wrinkles is easier than adding them, but de-aging requires imagining what someone looked like decades ago. Face shape, fat distribution, and features all change with age.
The Workaround:
- Keep de-aging subtle (10-15 years)
- Use reference photos if available
- Accept that significant de-aging will look artificial
- Hollywood uses teams of artists for this—it's not easy
Difficulty Rating: ⭐⭐⭐⭐ (Hard)
Child to Adult or Adult to Child
The Symptom: Proportions are completely wrong, face looks alien, features are distorted.
The Cause: Children's faces have fundamentally different proportions—larger eyes relative to face, smaller noses, different skull structure. These aren't just smaller adult faces.
The Workaround:
- Don't attempt it. The technology doesn't handle this well.
- If absolutely necessary, expect very poor results
Difficulty Rating: ⭐⭐⭐⭐⭐ (Extremely Hard)
Scenario: Facial Hair
Full Beards
The Symptom: Beard appears painted on, doesn't move with jaw, edges are wrong, texture is uniform instead of hair-like.
The Cause: Beards have complex 3D structure, cast shadows, and move independently. If source has a beard and target doesn't (or vice versa), the mismatch is severe.
The Workaround:
- Match facial hair between source and target
- If adding a beard, expect poor quality
- Consider physical beard application for real filming
Difficulty Rating: ⭐⭐⭐⭐⭐ (Very Hard when mismatched)
Stubble and 5 O'Clock Shadow
The Symptom: Stubble pattern is wrong, appears as discoloration rather than hair, inconsistent between frames.
The Cause: Stubble is fine detail that varies by individual. It's too detailed for current resolution handling and too individual to transfer between faces.
The Workaround:
- Match stubble levels between source and target
- Clean-shaven to clean-shaven works best
- Stubble to stubble is possible if similar
Difficulty Rating: ⭐⭐⭐ (Moderate when matched)
Mustaches and Partial Facial Hair
The Symptom: Mustache appears to float, doesn't connect properly to face, edges are blurred.
The Cause: Partial facial hair creates complex edge problems. Where does face end and hair begin? The system struggles with this boundary.
The Workaround:
- Match facial hair style closely
- Full coverage is often easier than partial
- Clean-shaven is easiest
Difficulty Rating: ⭐⭐⭐⭐ (Hard)
Scenario: Accessories and Modifications
Glasses
The Symptom: Frames flicker, lenses show wrong reflections, glasses float or clip through face, eyes behind glasses look strange.
The Cause: Glasses are physical objects on top of the face. They reflect the environment, create occlusion, and have their own geometry. The system doesn't understand them as objects—just as pixels.
The Workaround:
- Remove glasses from one or both subjects if possible
- Accept significant artifacts around glasses
- Post-processing can help with some issues
Difficulty Rating: ⭐⭐⭐⭐ (Hard)
Piercings
The Symptom: Piercings disappear, appear in wrong locations, flicker in and out.
The Cause: Small metal objects are treated as noise. They're too small to track reliably and too individual to transfer.
The Workaround:
- Remove piercings if possible before source capture
- Add piercings in post-processing
- Accept inconsistency
Difficulty Rating: ⭐⭐⭐ (Moderate)
Visible Tattoos (Face/Neck)
The Symptom: Tattoos are smeared, partially transferred, or completely removed.
The Cause: Tattoos are individual-specific patterns. They don't transfer between faces and are often treated as skin variation to be smoothed.
The Workaround:
- Match tattoo presence between source and target
- Consider covering tattoos for source capture
- Manual addition in post-processing
Difficulty Rating: ⭐⭐⭐⭐ (Hard)
Heavy Makeup
The Symptom: Makeup is transferred incorrectly, colors are wrong, contours don't make sense on new face, dramatic looks become distorted.
The Cause: Makeup changes face appearance significantly. The system may treat makeup as skin tone or facial features. Heavy contour especially confuses face geometry detection.
The Workaround:
- Use natural/minimal makeup for source capture
- Match makeup levels between source and target
- Add makeup in post-processing if needed
Difficulty Rating: ⭐⭐⭐ (Moderate to Hard depending on intensity)
Scenario: Distinctive Features
Scars
The Symptom: Scars disappear, appear in wrong locations, or are inconsistently visible.
The Cause: Scars are individual-specific and the system treats them as aberrations to be smoothed. They don't transfer between faces.
The Workaround:
- Match scar presence if possible
- Manual addition in post-processing
- Accept loss of distinctive scarring
Difficulty Rating: ⭐⭐⭐ (Moderate)
Highly Asymmetric Faces
The Symptom: Asymmetry is averaged out, face looks "normalized," distinctive character is lost.
The Cause: Models tend toward symmetry because it's more common in training data. Significant asymmetry is treated as error and corrected.
The Workaround:
- Use high-iteration processing to preserve asymmetry
- Match asymmetry direction between source and target
- Accept some loss of distinctive asymmetry
Difficulty Rating: ⭐⭐⭐ (Moderate)
Very Distinctive Bone Structure
The Symptom: Distinctive features (prominent cheekbones, unusual jaw, etc.) are muted or lost.
The Cause: The face is being mapped onto a different skull structure. Distinctive features are compressed or stretched to fit.
The Workaround:
- Match face shapes as closely as possible
- Accept that distinctive structure may be compromised
- Choose targets with similar basic geometry
Difficulty Rating: ⭐⭐⭐⭐ (Hard)
Quick Reference: Difficulty Summary
| Feature | Difficulty | Best Approach |
|---|---|---|
| Bald/shaved heads | ⭐⭐⭐ | Match source/target |
| Long flowing hair | ⭐⭐⭐⭐ | Minimize movement |
| Textured hairstyles | ⭐⭐⭐⭐ | Use diverse-trained models |
| Skin tone mismatch | ⭐⭐⭐⭐⭐ | Avoid large mismatches |
| Aging (large gap) | ⭐⭐⭐⭐⭐ | Keep changes small |
| Facial hair mismatch | ⭐⭐⭐⭐⭐ | Match between subjects |
| Glasses | ⭐⭐⭐⭐ | Remove if possible |
| Distinctive features | ⭐⭐⭐⭐ | Match source/target geometry |
Summary
Difficult facial features create specific, predictable problems. The common thread: features that are individual-specific, involve complex 3D structure, or differ significantly between source and target cause the most trouble.
The universal workaround is matching—source and target with similar features produce better results than mismatched pairs. When matching isn't possible, expect compromises and plan for post-processing.
Know your limitations before you start. Some combinations simply don't work well with current technology.
Related Topics
- Why Does My Deepfake Face Look Wrong? – Facial detail problems
- Why Do Deepfake Expressions Look Wrong? – Expression and motion challenges
- Why Do Deepfakes Still Look Wrong? Common Failure Modes – Feature-specific failures
- How Much Computing Power Does a Good Deepfake Need? – Quality trade-offs
- What Can't Deepfakes Do Yet? – Current technology limits