Skip to content

Latest commit

 

History

History
55 lines (51 loc) · 3.3 KB

README.md

File metadata and controls

55 lines (51 loc) · 3.3 KB

Input File Structure

Besides grouping input by class folders (refer to DisVoice), Face Mesh and HOG feature extraction requires an additional auxiliary data-enclosing folder within each class folder. Example:

facemesh/
├── EXPR_AGITATED
│   └── facecap
│       ├── agitated_sample_1.mp4
│       ├── agitated_sample_2.mp4
│       |── agitated_sample_3.mp4
|       ...
├── EXPR_RELIEVED
│   └── facecap
│       ├── relieved_sample_1.mp4
│       ├── relieved_sample_2.mp4
│       |── relieved_sample_3.mp4
|       ...
├── facedetection.py
├── facemesh_character.py
├── facemesh_features.md

Feature Extraction Usage

Face Mesh only provides the region of interest across frames. The HOG descriptor extract features from videos. Two strategies are implemented:

  1. Dense HOG: features are extracted per frame and either averaged or becomes inputs for feature late fusion
  2. Sparse HOG: space-time interest points12 across the video yields local features, pooled to video-level descriptors through Bag of Words

Use python facemesh_character.py --help for argument details

  • Input: must specify class names and the nested auxiliary directory as well
  • Configurations: option to use dense HOG over sparse HOG; variant selection for dense HOG; settings for sparse HOG.
  • Outputs:video labels & 1 of the following depending on extraction configurations
    • Video-level features from dense HOG averaged across fraems
    • Video-level features from sparse HOG
    • Frame-level features from dense HOG without average (must classify with feature late fusion)

Sparse HOG-Specific Dependencies

1. Space-Time Interest Point Detector

2. FFmpeg 0.5.2

Dependency of OpenCV 2.2.0

3. OpenCV 2.2.0

Dependency of Space-Time Interest Point Detector

  • Download source: https://sourceforge.net/projects/opencvlibrary/files/opencv-unix/2.2/
  • Python=2.7 for compatibility with OpenCV 2.2.0
  • Potentially needed OpenCV build (CMake overview) adjustments in CMakeLists.txt:
    • Line 214: set(OPENCV_BUILD_3RDPARTY_LIBS TRUE). Avoids backwards incompabtibility due to newer dependencies
    • Line 384: + set(ENV{PKG_CONFIG_PATH} "$ENV{PKG_CONFIG_PATH}:[FFmpeg 0.5.2 install location]/lib/pkgconfig"). Ensures desired FFmpeg installation will be used
    • Line 1275-6: target_compile_options(opencv_core PRIVATE -Wno-narrowing) & target_compile_options(opencv_ts PRIVATE -Wno-narrowing). Prevents build failure due to the narrowing conversion compiler warning
  • $ cmake with -DWITH_EIGEN2=OFF

Footnotes

  1. Laptev, I. On Space-Time Interest Points. Int J Comput Vision 64, 107–123 (2005). ↩

  2. https://github.com/theantimist/action-detection/ ↩