Skip to content

Commit

Permalink
bugfix of include.
Browse files Browse the repository at this point in the history
  • Loading branch information
wangzhaode committed Mar 7, 2024
1 parent 33932e0 commit 9f4d25b
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion src/tokenizer.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@
#include <codecvt>
#include <regex>
#include <set>
#include <climits>

// base64
static const std::string base64_chars =
Expand Down Expand Up @@ -638,4 +639,4 @@ std::string HuggingfaceTokenizer::decode(int id) {
r.push_back(char(u2b_.at(c)));
}
return r;
}
}

0 comments on commit 9f4d25b

Please sign in to comment.