Skip to content

Latest commit

 

History

History
155 lines (96 loc) · 5.76 KB

README.rst

File metadata and controls

155 lines (96 loc) · 5.76 KB

Tiling Pipeline

Single Scale Tiling

Single scale tiling FHD Gstreamer pipeline demonstrates splitting each frame into several tiles which are processed independently by hailonet element. This method is especially effective for detecting small objects in high-resolution frames.

Model

The VisDrone dataset consists of only small objects which can be assumed to be always confined within an single tile. As such it is better suited for running single-scale tiling with little overlap and without additional filtering.

Options

./tiling.sh [OPTIONS] [-i INPUT_PATH]
  • -i --input is an optional flag, a path to the video file displayed.
  • --print-gst-launch prints the ready gst-launch command without running it
  • --show-fps optional - enables printing FPS on screen
  • --tiles-x-axis optional - set number of tiles along x axis (columns)
  • --tiles-y-axis optional - set number of tiles along y axis (rows)
  • --overlap-x-axis optional - set overlap in percentage between tiles along x axis (columns)
  • --overlap-y-axis optional - set overlap in percentage between tiles along y axis (rows)
  • --iou-threshold optional - set iou threshold for NMS.
  • --sync-pipeline optional - set pipeline to sync to video file timing.

Run

cd $TAPPAS_WORKSPACE/apps/h8/gstreamer/general/tiling
./tiling.sh

The output should look like:

Method of Operation

This app is based on our tiling pipeline template

Multi Scale Tiling

Multi-scale tiling FHD Gstreamer pipeline demonstrates a case where the video and the training dataset includes objects in different sizes. Dividing the frame to small tiles might miss large objects or “cut" them to small objects. The solution is to split each frame into number of scales (layers) each includes several tiles.

Multi-scale tiling strategy also allows us to filter the correct detection over several scales. For example we use 3 sets of tiles at 3 different scales:

  • Large scale, one tile to cover the entire frame (1x1)
  • Medium scale dividing the frame to 2x2 tiles.
  • Small scale dividing the frame to 3x3 tiles.

In this mode we use 1 + 4 + 9 = 14 tiles for each frame. We can simplify the process by highlighting the main tasks: Crop -> Inference -> Ppost-process -> Aggregate → Remove exceeded boxes → Remove large landscape → Perform NMS

Model

Options

./multi_scale_tiling.sh [OPTIONS] [-i INPUT_PATH]
  • -i --input is an optional flag, a path to the video file displayed.
  • --print-gst-launch prints the ready gst-launch command without running it
  • --show-fps optional - enables printing FPS on screen
  • --tiles-x-axis optional - set number of tiles along x axis (columns)
  • --tiles-y-axis optional - set number of tiles along y axis (rows)
  • --overlap-x-axis optional - set overlap in percentage between tiles along x axis (columns)
  • --overlap-y-axis optional - set overlap in percentage between tiles along y axis (rows)
  • --iou-threshold optional - set iou threshold for NMS.
  • --border-threshold optional - set border threshold to Remove tile's exceeded objects.
  • --scale-level optional - set scales (layers of tiles) in addition to the main layer. 1: [(1 X 1)] 2: [(1 X 1), (2 X 2)] 3: [(1 X 1), (2 X 2), (3 X 3)]]'

Run

cd $TAPPAS_WORKSPACE/apps/h8/gstreamer/general/tiling
./multi_scale_tiling.sh

The output should look like:

Mode of Operation

As multi scale tiling is almost equal to single scale, the differences are listed below:

TILE_CROPPER_ELEMENT="hailotilecropper internal-offset=$internal_offset name=cropper tiling-mode=1 scale-level=$scale_level

hailotilecropper sets tiling-mode to 1 (0 - single-scale, 1 - multi-scale) and scale-level to define what is the structure of scales/layers in addition to the main scale.

hailonet hef-path is mobilenet_ssd which is training dataset includes objects in different sizes.

   hailotileaggregator flatten-detections=true iou-threshold=$iou_threshold border-threshold=$border_threshold name=agg

``hailotileaggregator`` sets ``border-threshold`` used in remove tile's exceeded objects process.

Using Retraining to Replace Models

Note

It is recommended to first read the Retraining TAPPAS Models page.

Retraining Dockers (available on Hailo Model Zoo), can be used to replace the following models with ones that are trained in the dataset:

  • mobilenet_ssd
    • Retraining docker
    • TAPPAS changes to replace model:
      • Update HEF_PATH on the .sh file
      • Update mobilenet_ssd.cpp with the new paremeters, then recompile to create libmobilenet_ssd_post.so