New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat - add a vision at a clip universe subfolder #23

Open

rhysdg opened this issue Aug 10, 2024 · 0 comments

Assignees

Labels

Owner

rhysdg commented Aug 10, 2024 •

edited

Loading

House a number of examples that don't necessarily fit in the pure onnxruntime setting
Llava-next with mistral 7b for instance works really well on a single GPU, 4-bit quant, with ray serve and huggingface - adding example shortly. Noting that really well means it fits on a single GPU, it's still chunky so high latency on a consumer GPu at least

rhysdg added the enhancement label

rhysdg self-assigned this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment