Recipe for TensorRT plugins

[bcis93]

Hi All,

I'm trying to build libnvinfer_plugin.so using the open source TensorRT, so I created the recipe below. It successfully builds the library, but then I get the following issue during the final do_rootfs step since this library is already provided by tensorrt_7.1.3-1.bb

Error: Transaction check error:
  file /usr/lib/libnvinfer_plugin.so.7 conflicts between attempted installs of libnvinfer-plugin7-7.1.3+1-r0.armv8a_tegra186 and tensorrt-7.1.3+1-r0.armv8a_tegra
  file /usr/lib/libnvinfer_plugin.so.7.1.3 conflicts between attempted installs of libnvinfer-plugin7-7.1.3+1-r0.armv8a_tegra186 and tensorrt-7.1.3+1-r0.armv8a_tegra

How should I handle the conflict here? What is the best way to use the libnvinfer_plugin.so built from open source instead of using the one provided by the tensorrt recipe?

Here's the recipe I created for reference:

SUMMARY = "TensorRT Open source libnvinfer_plugin"
HOMEPAGE = "http://developer.nvidia.com/tensorrt"
LICENSE = "Apache-2.0"
LIC_FILES_CHKSUM = "file://LICENSE;md5=b43e4a60a0643023327e7caf2dbf8663"

DEPENDS = "zlib libcublas cudnn tensorrt cuda-cudart"

inherit cmake cuda

SRC_REPO = "github.com/NVIDIA/TensorRT.git;protocol=https"
SRCBRANCH = "master"
SRC_URI = "gitsm://${SRC_REPO};branch=${SRCBRANCH} \
           file://0001-Use-CUDA_TOOLKIT_TARGET_DIR.patch \
           "

SRCREV = "9a9cae75e7155b2114454f37ccc49eca9d3352dc"

S = "${WORKDIR}/git"

EXTRA_OECMAKE = "-DGPU_ARCHS=${@extract_sm(d)} -DBUILD_PARSERS=OFF -DBUILD_SAMPLES=OFF"
OECMAKE_TARGET_COMPILE = "nvinfer_plugin"

CXXFLAGS += "${CUDA_CXXFLAGS}"

do_install() {
    install -d ${D}${libdir}
    cp --preserve=mode,timestamps  ${B}/libnvinfer_plugin.so* ${D}${libdir}
}

# Without this I get:
# "%package -n libnvinfer-plugin7: package libnvinfer-plugin7 already exists"
# at the rpm creation step.
FILES_${PN}-dev = ""
FILES_${PN} = "${libdir}/*"

Thanks!

[ichergui]

Hey @bcis93

You can add tensorrt_7.1.3-1.bbappend and add do_install_append to remove the library from there to avoid this conflict.

Are you using the same recipe name as the original one that is provide by meta-tegra ?

I can try it and will let you know.

[bcis93]

Hey @ichergui, yes, a .bbappend would definitely work, but I was actually hoping to do this within meta-tegra and add my new recipe for anyone who also wants to build libnvinfer_plugin.so from source.

I have thought about using update-alternatives to handle the conflict between the two recipes, or I’ve also thought about adding a new package to the tensorrt recipe that doesn’t install that library. I’m just trying to figure out the cleanest/most usable approach!

[ichergui]

Hey @bcis93
The bbapend was just a workaround for your test. I had a discussion with Matt and the outcome is we want to introduce the recipe in our tegra meta layer. However, we thought to have a possibility to enable more than one plugin via PACKAGECONFIG.

I'm planing to experiment this over the weekend, I will keep you updated.

[bcis93]

@ichergui sounds great! Let me know what I can do to help!

[ichergui]

Hello @bcis93
Please take a look on the PR here #799

[bcis93]

Hey @ichergui, thanks for getting that PR started! I'm building off the dunfell-l4t-r32.4.3 branch, so I copied what you did over to my own branch and built tensorrt 7.1.3 instead of 8.0.1. You can see what I did here (but it's basically the exact same as the changes in your PR).

To build the plugin, I added PACKAGECONFIG_pn-tensorrt="plugin" to my .conf file for my distro, and it compiles the plugin just fine, but I'm again hitting the same error:

Error: Transaction check error:
  file /usr/lib/libnvinfer_plugin.so.7.1.3 conflicts between attempted installs of libnvinfer-plugin7-7.1.3-r0.armv8a_tegra186 and tensorrt-prebuilt-7.1.3+1-r0.armv8a_tegra

Any thoughts on this? Did you see this conflict as well?

[ichergui]

Hello @bcis93
Yeah, I see this conflict when trying to install tensorrt.
This happen because of the dependency with tensorrt-prebuilt. I have to rework a bit the cross compilation build to avoid dependency with the tensorrt-prebuilt
I will keep you updated

[bcis93]

@ichergui sounds good, just wanted to make sure I was seeing the same thing as you.

But I think we're still going to need tensorrt-prebuilt, right? Because the open source TensorRT repo will build the plugins, parsers, and samples, but it doesn't build everything. From the Readme in that repo:

This repository contains the Open Source Software (OSS) components of NVIDIA TensorRT. Included are the sources for TensorRT plugins and parsers (Caffe and ONNX), as well as sample applications demonstrating usage and capabilities of the TensorRT platform. These open source software components are a subset of the TensorRT General Availability (GA) release with some extensions and bug-fixes.

[ichergui]

Hello @bcis93
Sorry, I couldn't reply yesterday, I wasn't available due my travelling to France.
I have rework the recipes and fix build issues with tegra-mmapi-samples -> video_dec_trt. Thanks to @madisongh for his help.
Please take a look on the updates #799

Note: The tensorrt-plugins should provide the same plugins as tensorrt-plugins-prebuilt or more.

Kindly find the packages contents (tensorrt-plugins-prebuilt and tensorrt-plugins):

• tensorrt-plugins-prebuilt

➜  build-xavier-nx-devkit git:(master) tree -s tmp/work/armv8a_tegra-oe4t-linux/tensorrt-plugins-prebuilt/8.0.1-1-r0/packages-split/tensorrt-plugins-prebuilt 
tmp/work/armv8a_tegra-oe4t-linux/tensorrt-plugins-prebuilt/8.0.1-1-r0/packages-split/tensorrt-plugins-prebuilt
└── [       4096]  usr
    ├── [       4096]  lib
    │   ├── [         26]  libnvinfer_plugin.so.8 -> libnvinfer_plugin.so.8.0.1
    │   ├── [   18280576]  libnvinfer_plugin.so.8.0.1
    │   ├── [         24]  libnvonnxparser.so.8 -> libnvonnxparser.so.8.0.1
    │   └── [    2670752]  libnvonnxparser.so.8.0.1
    └── [       4096]  src
        └── [       4096]  tensorrt
            └── [       4096]  bin
                └── [     365832]  trtexec

5 directories, 5 files
➜  build-xavier-nx-devkit git:(master) 

• tensorrt-plugins

➜  build-xavier-nx-devkit git:(master) tree -s tmp/work/armv8a_tegra194-oe4t-linux/tensorrt-plugins/8.0.1-1-r0/packages-split/tensorrt-plugins 
tmp/work/armv8a_tegra194-oe4t-linux/tensorrt-plugins/8.0.1-1-r0/packages-split/tensorrt-plugins
└── [       4096]  usr
    └── [       4096]  lib
        ├── [         25]  libnvcaffeparser.so.8 -> libnvcaffeparser.so.8.0.1
        ├── [     928384]  libnvcaffeparser.so.8.0.1
        ├── [         26]  libnvinfer_plugin.so.8 -> libnvinfer_plugin.so.8.0.1
        ├── [   11811536]  libnvinfer_plugin.so.8.0.1
        ├── [         24]  libnvonnxparser.so.8 -> libnvonnxparser.so.8.0.1
        └── [     802328]  libnvonnxparser.so.8.0.1

2 directories, 6 files
➜  build-xavier-nx-devkit git:(master) 

• All samples are built when using the tensorrt-plugins (build tensorrt plugins from source)

➜  build-xavier-nx-devkit git:(master) tree -s tmp/work/armv8a_tegra194-oe4t-linux/tensorrt-plugins/8.0.1-1-r0/packages-split/tensorrt-plugins-samples 
tmp/work/armv8a_tegra194-oe4t-linux/tensorrt-plugins/8.0.1-1-r0/packages-split/tensorrt-plugins-samples
└── [       4096]  usr
    └── [       4096]  bin
        ├── [      80272]  sample_algorithm_selector
        ├── [     162184]  sample_char_rnn
        ├── [      84376]  sample_dynamic_reshape
        ├── [      80264]  sample_fasterRCNN
        ├── [      63880]  sample_googlenet
        ├── [      88448]  sample_int8
        ├── [     108616]  sample_int8_api
        ├── [      84352]  sample_mlp
        ├── [      67976]  sample_mnist
        ├── [      80264]  sample_mnist_api
        ├── [     227448]  sample_nmt
        ├── [      67976]  sample_onnx_mnist
        ├── [      67992]  sample_onnx_mnist_coord_conv_ac
        ├── [      88464]  sample_reformat_free_io
        ├── [      96640]  sample_ssd
        ├── [     109072]  sample_uff_fasterRCNN
        ├── [      84360]  sample_uff_maskRCNN
        ├── [      67976]  sample_uff_mnist
        ├── [      88464]  sample_uff_plugin_v2_ext
        ├── [      92552]  sample_uff_ssd
        └── [     264264]  trtexec

2 directories, 21 files
➜  build-xavier-nx-devkit git:(master) 

Still improvements that we can apply to avoid dependency with Protobuf. I will keep you updated.
Once the PR will be merged, I will backport these changes to the other branches.

[ichergui]

Hello @bcis93 @madisongh

I have run the sample applications mentioned in my previous post.

I copied manually data/models to my device Jetson Xavier NX (eMMC version) for my tests. These data/models came from the deb package libnvinfer-samples_8.0.1-1+cuda10.2_all.deb provided by Nvidia.

Note: We have to download some of the data/model via bash or python script that we can find in TensorRT --> samples

Kindly find the results of my tests in the attached files.

trt_sample_algorithm_selector.log
trt_sample_char_rnn.log
trt_sample_dynamic_reshape.log
trt_sample_fasterRCNN.log
trt_sample_googlenet.log
trt_sample_int8_api.log
trt_sample_mlp.log
trt_sample_mnist.log
trt_sample_mnist_api.log
trt_sample_onnx_mnist.log
trt_sample_reformat_free_io.log
trt_sample_ssd_KO.log
trt_sample_uff_mnist.log
trt_sample_uff_plugin_v2_ext.log
trt_sample_uff_ssd_KO.log
trt_trtexec_onnx_model_with_static_input_shapes.log
trt_trtexec_simple_mnist_model_caffe.log
trt_trtexec_tune_throughput_with_multi-streaming.log

I will continue my work to make the same data/models available for both tensorrt-plugins-prebuilt and tensorrt-plugins (build tensorrt plugins from source).
I will keep you guys updated

[bcis93]

Thanks @ichergui! I'll test this out as well today to make sure it works for me, but this looks great!

[bcis93]

@ichergui I had a chance to test this today, and it's looking good!

Once the PR will be merged, I will backport these changes to the other branches.

I'm working on the dunfell-l4t-r32.4.3 branch, so I went ahead and got the changes started for that one. I opened PR #800 for that, and we can update that branch/PR as needed if there are any other changes we need here.

[ichergui]

Hey @bcis93
The PR #799 is merged, you can go ahead to backport it to L4T R-32.4.3
Please let me know if you need help

[bcis93]

Thanks for your help @ichergui! I just backported it and marked the PR ready for review.