[Fix] Add fix for #92: Support `class.method` as a Tokenizer for `CustomEmbedding` +. minor changes#128

Merged

bhavnicksm merged 9 commits intomainfrom development

Jan 2, 2025

+117-21

Commits on Dec 31, 2024

[fix] #93 : Support passing a callable which may be a function or a method for token counting
bhavnicksm
committed

Commits on Jan 2, 2025