Skip to content

Latest commit

 

History

History
167 lines (112 loc) · 5.68 KB

convert_dynamic.md

File metadata and controls

167 lines (112 loc) · 5.68 KB

Inference - Model Shape Scaling

1. Introduction

In some inference scenarios, such as object recognition after detection, the input batch size and image size of the recognition network are not fixed because the number of object and the size of the object are not fixed. If each inference is calculated according to the maximum batch size or maximum image size, it will cause a waste of computing resources.

Therefore, you can set some candidate values during model conversion, and resize to the best matching candidate value during inference, thereby improving performance. Users can manually select these candidate values empirically, or they can be statistically derived from the dataset.

This tool integrates the function of dataset statistics, can count the appropriate combination of batch size, height and width as candidate values, and encapsulates the model conversion tool, thus realizing the automatic model shape scaling.

2. Environment

Please refer to Environment Installation to install ACL or MindSpore Lite environment.

3. Model

Currently, ONNX model files are supported, and by selecting different backends, they are automatically shape scaling and converted to OM or MIndIR model files.

Please make sure that the input model is the dynamic shape version. For example, if the text detection model needs to shape scaling for H and W, make sure that at least the H and W axes are dynamic, and the shape can be (1,3,-1,-1) and (-1,3,- 1,-1) etc.

4. Dataset

Two types of data are supported:

  1. Image folder

    • This tool will read all the images in the folder, record height and width, and count suitable candidate values

    • Suitable for text detection and text recognition models

  2. Annotation file for text detection

    • Refer to converter, which is the annotation file output when the parameter task is det

    • This tool will read the coordinates of the text box marked under each image, record height and width, and the number of boxes as batch size, and count suitable candidate values

    • Suitable for text recognition models

5. Usages

cd deploy/models_utils/auto_scaling

5.1 Command example
  • auto shape scaling for batch size
python converter.py \
    --backend=atc \
    --model_path=/path/to/model.onnx \
    --dataset_path=/path/to/det_gt.txt
    --input_shape=-1,3,48,192 \
    --output_path=output

The output is a single OM model: model_dynamic_bs.om

  • auto shape scaling for height and width
python converter.py \
    --backend=atc \
    --model_path=/path/to/model.onnx \
    --dataset_path=/path/to/images \
    --input_shape=1,3,-1,-1 \
    --output_path=output

The output is a single OM model: model_dynamic_hw.om

  • auto shape scaling for batch szie, height and width
python converter.py \
    --backend=atc \
    --model_path=/path/to/model.onnx \
    --dataset_path=/path/to/images \
    --input_shape=-1,3,-1,-1 \
    --output_path=output

The output result is multiple OM models, combining multiple different batch sizes, and each model uses the same dynamic image size：model_dynamic_bs1_hw.om, model_dynamic_bs4_hw.om, ......

  • no shape scaling
python converter.py \
    --backend=atc \
    --model_path=/path/to/model.onnx \
    --input_shape=4,3,48,192 \
    --output_path=output

The output is a single OM model: model_static.om

5.2 Parameter Details
Name Default Required Description
model_path None Y Path to model file
input_shape None Y model input shape, NCHW format
data_path None N Path to image folder or annotation file
input_name x N model input name
backend atc N converter backend，atc or lite
output_path ./output N Path to output model
soc_version Ascend310P3 N soc_version for Ascend，Ascend310P3 or Ascend310
5.3 Configuration file

In addition to the above command line parameters, there are some parameters in auto_scaling.yaml to describe the statistics of the dataset, You can modify it yourself if necessary:

  • limit_side_len

    The size limit of height and width of the original input data, if it exceeds the range, it will be compressed according to the ratio, and the degree of discreteness of the data can be adjusted.

  • strategy

    Data statistics algorithm strategy, supports mean_std and max_min two algorithms, default: mean_std.

    • mean_std
    mean_std = [mean - n_std * sigma，mean + n_std * sigma]
    • max_min
    max_min = [min - (max - min) * expand_ratio / 2，max + (max - min) * expand_ratio / 2]

  • width_range/height_range

    For the width/height size limit after discrete statistics, exceeding will be filtered.

  • interval

    Interval size, such as some networks may require that the input size must be a multiple of 32.

  • max_scaling_num

    The maximum number of discrete values for shape scaling .

  • batch_choices

    The default batch size value, if the data_path uses an image folder, the batch size information cannot be counted, and the default value will be used.

  • default_scaling

    If user does not use data_path, provide default height and width discrete values for shape scaling .