Replace Cached with Yolox-graphic-elements and drop Deplot #365
Conversation
…v/nv-ingest into edwardk/replace-cached-with-yolox-1
…v/nv-ingest into edwardk/replace-cached-with-yolox-1
| capabilities: [gpu] | ||
| runtime: nvidia | ||
|
|
||
| deplot: | ||
| image: ${DEPLOT_IMAGE:-nvcr.io/nvidia/nemo-microservices/deplot}:${DEPLOT_TAG:-1.0.0} | ||
| yolox-graphic-elements: |
There was a problem hiding this comment.
I might be misreading GitHub's review changes UI, but can we put this new NIM on device 1, since 0 is overloaded?
There was a problem hiding this comment.
We used to have Deplot on device 0, but we can remove Deplot with this PR, so I moved everything to device 0 including this new NIM. Isn't that what we wanted to do by dropping Deplot, i.e., fit everything in 1 device and drop the minimum requirement to 1 GPU?
There was a problem hiding this comment.
that is one goal, but we should do that subject to more benchmarking. we don't yet have the ability to set the batch size, and I'd like to confirm we're not saturating compute capacity of a single device (for good portion of bo767 execution, both GPUs have >50% utilization).
There was a problem hiding this comment.
ok, moved the new nim to device 1 in b5fd7fa.
Description
Checklist