Skip to content

Face Recognition

The Face Recognition batch tool analyzes video thumbnails to automatically detect and tag actors using face recognition. You can access this tool from the start page or by right-clicking a video and selecting Detect Actors... from the context menu.

The tool offers three operation modes:

Learn from existing cast -- Improves recognition accuracy by scanning videos where actors have already been manually tagged. The system detects faces in the video thumbnails and matches them against each actor's existing embeddings. When a strong match is found, it saves that face as an additional embedding for the actor. This is useful because faces captured from real video footage (with natural lighting, angles, and resolution) complement the companion photo embeddings and help the system recognize actors more reliably. Prerequisite: Actors must already have at least one embedding from companion images before this mode can work -- actors with no embeddings are skipped.

Detect and add actors to cast -- Scans video thumbnails for faces and automatically adds matching actors to the cast. This is the primary mode for batch face detection.

Remove thumbnails without faces -- Deletes thumbnails where no face is detected in the image.

You can select which videos to process: All videos, the currently selected videos, or the filtered set. You can also choose to skip videos that already have actors in their cast to avoid redundant processing. The frame skip setting controls how many thumbnails to skip between processed frames, where 1 means every frame is processed and higher values are faster.

How It Works

The face recognition system uses a 128-dimensional "embedding" as a unique fingerprint for each face. The process works as follows:

  1. Faces are extracted from video thumbnails captured during indexing
  2. A 128-dimensional embedding is generated for each detected face
  3. These embeddings are compared against known actor embeddings
  4. If a match exceeds the confidence threshold and the actor appears in multiple frames, they are tagged

The requirement for multiple frame appearances helps reduce false positives -- an actor must be consistently recognized across different thumbnails to be tagged.

Generating Face Embeddings

Before the system can recognize an actor, it needs to learn their face by generating face embeddings from companion images.

Learning faces from companion images

  1. Open the Edit Actor dialog (double-click an actor)
  2. Add companion images showing the actor's face
  3. Select one or more companion images in the list
  4. Click Learn Face to generate embeddings from the selected images
  5. The embedding count is shown in the Companion Images header, e.g. "(8 face embeddings)"
  6. Images that have been used for learning show a green border

You can also right-click a single companion image and select Learn face from this image.

Tips for best results

  • Select 8 high quality images with clear faces, in different lighting and different angles
  • Use images with only one person for the first image -- when there are no existing embeddings, the system cannot determine which face belongs to the actor if multiple people are in the image
  • Use clear, well-lit photos where the face is the main subject
  • Include a variety of expressions and poses
  • Avoid sunglasses or heavy makeup that obscures features

The system matches faces in subsequent images against existing embeddings to ensure it's learning the correct person. If multiple faces are detected and no existing embeddings exist, the image is skipped.

Managing Embeddings

To see how many embeddings an actor has, open the Edit Actor dialog -- the count is shown in the Companion Images expander header, e.g. "(8 face embeddings)".

To clear all embeddings for an actor, click Clear All in the Face Recognition section of the Edit Actor dialog. This removes all learned face data for that actor. When you delete an actor, their embeddings are automatically removed.

Detection Methods

Two face detection methods are available:

HOG (Histogram of Oriented Gradients)

  • Faster processing
  • Works well for clear, front-facing shots
  • May miss faces at angles or in poor lighting
  • Good for quick scans of large batches

CNN (Convolutional Neural Network)

  • Slower but more accurate
  • Better at finding faces at various angles
  • More robust to lighting variations
  • Recommended for thorough detection

You can select the detection method in the Face Recognition preferences.

Tips for Best Results

  1. Quality companion images matter -- Use clear, well-lit photos where the face is prominent
  2. Add multiple embeddings -- Different angles and lighting conditions improve robustness. 8 good embeddings per actor is recommended
  3. More thumbnails = better detection -- Videos with more thumbnails give more chances to detect faces. Consider increasing thumbnail count for videos with important actors
  4. Use companion images wisely -- The green border shows which images have been used for learning
  5. Review results -- Face recognition isn't perfect. Occasionally review auto-tagged actors for accuracy and remove incorrect tags

See the Face Recognition preferences for confidence thresholds and other settings.

Troubleshooting

"No faces detected in image"

  • The image may be too small -- try a larger, higher resolution image
  • The face may be partially obscured or at an extreme angle
  • Try a different photo with a clearer view of the face

"Actors not being detected in videos"

  • Ensure the actor has at least one embedding (check the Edit Actor dialog)
  • The video thumbnails may not show the actor's face clearly
  • Try adding more embeddings from different angles
  • Verify the actor appears in enough frames (check the minimum frames setting)

"Too many false positives"

  • Add more embeddings to help distinguish between similar faces
  • Remove embeddings generated from low-quality source images
  • Consider using the CNN detection method for better accuracy

"Detection is very slow"

  • CNN method is slower but more accurate -- switch to HOG for large batches
  • Processing time depends on the number of thumbnails per video

Automatic Faceless Thumbnail Removal

You can configure the indexing pipeline to automatically remove thumbnails where no face is detected. Enable Remove thumbnails without faces in the Video Indexer preferences tab. This runs after all analysis (face recognition, scene classification, object detection) is complete, so you can also enable But keep if keyworded to preserve faceless thumbnails that have scene or object classifications. This setting does not apply when capturing a single frame.

For best results with face detection, use a thumbnail width of at least 320 pixels (set in the Video Indexer preferences).