From 2f59c5fc3b43b46119c4458d3860de0b16a58d48 Mon Sep 17 00:00:00 2001 From: Rhys Williams Date: Thu, 20 Jun 2024 22:12:33 -0400 Subject: [PATCH] adding hidden clariification --- README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/README.md b/README.md index bda4867..297af53 100644 --- a/README.md +++ b/README.md @@ -79,12 +79,11 @@ Last of all the aim here is to keep up with the latest optimised foundation mode - Notice also cosine similrity at `get_similarity_scores` is adusted to handle multiple context - in other words a handful of text embedding can be sent as 'contexts', and send to the function to be evaluated against a single image or a batch of images. -- hidden states are also available at `onnx_model.hidden_image` and `onnx_model.hidden_text` allowing for analysis, attention plotting and multi-point processing as input to SAM. Watch this space for more on this. The stock implementation below uses the second `pooler` output for each in a cosine similarity scenario. +- hidden states are also available at `onnx_model.hidden_image` and `onnx_model.hidden_text` when using `type=siglip` for extraction only - allowing for analysis, attention plotting and multi-point processing as input to SAM. Watch this space for more on this. - Not also that an `OnnxSAM` class is also available with the same instantiation and automatic model download - further examples are on their way along with SigLIP integration - ## Example usage (CLIP/SigLIP - SAM incoming) : - For the full 384 SigLIP model go ahead and use the `.inference` method as follows. Noting that CLIP is avaiable via the same method. Either model will switch between softmax and sigmoid accordingly: