Common API

[attila-dusnoki-htec]

The main idea is to come upt with a solution to mimic what HIP does for CUDA.

Ideally, we could run any TRT apps with MGX without any code change. But we could also do something similar what hipify does to convert the code to a Common API.

[mirza-halilcevic]

I created a separate issue for the initial stage: #187

[mirza-halilcevic]

We started working on a C++ API equivalent to TensorRT's API. The only difference between the APIs, for now, is the namespaces (nvinfer1 -> mgxinfer1 and nvonnxparser -> mgxonnxparser). We can consider even keeping the same namespace later on. The code can be found here. At the moment, the new API is just a wrapper around the existing MIGraphX API. No modification of MIGraphX internals was needed. With this, we can successfully run the MNIST TensorRT sample.

Closed #187

[mirza-halilcevic]

TensorRT has specific object lifetime rules. Some classes are factories for others and need to outlive them. Some return pointers to internal objects whose lifetime is not managed by the user. For now, we don't focus too much on these rules and use smart pointers to ensure no dangling references until we get the functionality right.

[mirza-halilcevic]

Dynamic shapes are handled differently between the two APIs. TensorRT expects the dynamic shapes after parsing, but before compilation, whereas MIGraphX expects the shapes during parsing. A workaround would be to replace the specific instructions after parsing. As discussed with the AMD team, the dynamic shapes support in MIGraphX is currently lacking, so we'll focus on that a bit later.

[mirza-halilcevic]

We started implementing the API related to manual network creation. The goal is to get sampleCharRNN to run successfully. Related issue: #188

The TensorRT ONNX parser is open source with an Apache 2.0 license. This could prove useful later on with more complicated scripts that modify the network after creating it from an ONNX file.