Code

Fix usage of whitespace tokenizer#410

Code

Merged

indykoning merged 2 commits intomasterrapidez/core:masterfrom tokenizer-lowercaserapidez/core:tokenizer-lowercaseCopy head branch name to clipboardJan 29, 2024

+1-1

Commits

Commits on Jan 25, 2024

• Use lowercase tokenizer

  Jade-GG

  authored
  6048413

  Copy full SHA for 6048413

Commits on Jan 29, 2024

• Use standard tokenizer instead

  Jade-GG

  authored
  18cc786

  Copy full SHA for 18cc786