Test & Maybe Release

feat: export `Tokenizer`, document how it can be used #300

Sign in to view logs

Run time

Learn about OS pricing on GitHub Actions

Job	Run time
test (16.x, macos-latest)	1m 38s
test (16.x, ubuntu-latest)	44s
test (16.x, windows-latest)	1m 32s
test (18.x, macos-latest)	1m 27s
test (18.x, ubuntu-latest)	53s
test (18.x, windows-latest)	1m 51s
test (lts/*, macos-latest)	1m 23s
test (lts/*, ubuntu-latest)	47s
test (lts/*, windows-latest)	1m 12s
test (current, macos-latest)	1m 34s
test (current, ubuntu-latest)	42s
test (current, windows-latest)	1m 18s
Release	0s
	15m 1s