Skip to content

Latest commit

 

History

History
118 lines (79 loc) · 5.1 KB

README.rst

File metadata and controls

118 lines (79 loc) · 5.1 KB

Face Recognition Pipeline

Overview

face_recognition.sh demonstrates face recognition pipeline on a video stream.

Options

./face_recognition.sh
  • -i --input is an optional flag, a path to an input source (video file / camera device)
  • --print-gst-launch prints the ready gst-launch command without running it
  • --show-fps is an optional flag that enables printing FPS on screen
  • --network to set which network to use. choose from [scrfd_10g, scrfd_2.5g], default is scrfd_10g"

Models

Run

cd $TAPPAS_WORKSPACE/apps/gstreamer/general/face_recognition/
./face_recognition.sh

The output should look like:

How does it work?

The pipeline is devided to 4 steps:

  1. Face detection and landmarks:
    Detect faces and predict the locations of key facial landmarks (such as eyes, nose, and mouth), in the video stream. The scrfd_10g network is more accurate but computationally heavier, while the scrfd_2.5g network is less accurate but more lightweight, providing better performance.
  2. Face alignment:
    This step involves using the detected landmarks and the original video frame to compute an affine transformation that aligns the face with a predefined destination matrix. This ensures that the face is consistently positioned in the same way for the next step in the pipeline.
  3. Embedding matrix:
    Run Arcface network to generate an embedding matrix for each aligned face. An embedding is a compact representation of the face that captures its unique characteristics. This embedding can then be compared to other embeddings to determine the similarity between faces.
  4. Gallery:
    Use the generated embeddings to find the closest matching face in the local database (named "Local Gallery" and stored in a JSON file). This allows the application to identify the person in the video stream.

How to save faces to the local gallery?

The local gallery file face_recognition_local_gallery.json is stored under apps/gstreamer/general/face_recognition/resources/gallery directory. It contains the embeddings of the faces.

To add faces to the gallery, you can use the save_face.sh script.

cd $TAPPAS_WORKSPACE/apps/gstreamer/general/face_recognition/
./save_faces.sh

Options:

  • --network to set which network to use. choose from [scrfd_10g, scrfd_2.5g], default is scrfd_10g
  • --clean to clean the local gallery file enitrely

The script goes over the .png files in resources/faces directory, and saves each face into the gallery. The name of the face is determined by the file name.

To use your own video sources and faces, add your images to the resources/faces directory and remove the original ones. Make sure to use .png format image files and a file name including the name of the person. Also use --clean option to order the script to clean the gallery file before saving the new faces.

How to use Retraining to replace models

Note

It is recommended to first read the Retraining TAPPAS Models page.

You can use Retraining Dockers (available on Hailo Model Zoo), to replace the following models with ones that are trained on your own dataset:

  • scrfd_10g
    • No retraining docker is available.
    • Post process CPP file edit update post-processing:
      • Update face_detection.cpp (scrfd() fucttion) with your new paremeters, then recompile to create libface_detection_post.so
  • scrfd_2.5g
    • No retraining docker is available.
    • Post process CPP file edit update post-processing:
      • Update face_detection.cpp (scrfd() fucttion) with your new paremeters, then recompile to create libface_detection_post.so
  • arcface_mobilefacenet
    • Retraining docker
    • TAPPAS changes to replace model:
      • Update HEF_PATH on the .sh file
      • Update arcface.cpp with your new paremeters, then recompile to create libface_recognition_post.so