Models pylaia pylaia belfort

pylaia/belfort/README.md

@@ -0,0 +1,26 @@
+# PyLaia Belfort
+
+## Datasets
+
+This model is trained from scratch on the [Belfort](https://demo.arkindex.org/browse/d6fe867e-d3dc-4a40-b5e8-c54b08f79137) corpus:
+* PyLaia is trained on ROVER annotations created from [Callico](https://callico.teklia.com/projects/ce9b42d4-23a8-4381-b5bb-459bedc59165/details/), [DAN](https://demo.arkindex.org/process/worker-version/d40bdd30-1913-469c-a0ff-4f49558e59b6), [PyLaia](https://demo.arkindex.org/process/worker-version/155854f5-21e3-4d60-9d81-d6d2cbd9ddee) transcriptions
+* The ARPA language model is trained on Callico annotations
+
+The following (random) splits are used:
+* [train set](https://demo.arkindex.org/element/3cd76b7a-af1a-436b-8d81-e9fa6d62323b),
+* [validation set](https://demo.arkindex.org/element/ddd9979f-3fdb-4dbc-bdb1-1e0dc9577a3b),
+* [test set](https://demo.arkindex.org/element/93f02102-23bc-477d-8b96-701ce92487d0).
+
+The dataset is available in `/home/training_data/ATR_line/Belfort/belfort_final/`.
+
+## Results
+
+* Support: 19751 (train) / 2770 (validation) / 2490 (test)
+* Best LM weight: 2.0
+* Redmine issue [#2531](https://redmine.teklia.com/issues/2531)
+
+| Model           | val CER | val WER | test CER | test WER |
+| --------------- | ------- | ------- | -------- | -------- |
+| PyLaia          | 8.78    | 23.54   | 7.96     | 23.25    |
+| PyLaia + 6-gram | 8.06    | 19.63   | 7.34     | 19.78    |
+

pylaia/belfort/lexicon.txt

@@ -0,0 +1,112 @@
+<ctc> <ctc>
+" "
+% %
+& &
+' '
+( (
+) )
+* *
+, ,
+- -
+. .
+/ /
+0 0
+1 1
+2 2
+3 3
+4 4
+5 5
+6 6
+7 7
+8 8
+9 9
+: :
+; ;
+< <
+= =
+> >
+? ?
+A A
+B B
+C C
+D D
+E E
+F F
+G G
+H H
+I I
+J J
+K K
+L L
+M M
+N N
+O O
+P P
+Q Q
+R R
+S S
+T T
+U U
+V V
+W W
+X X
+Y Y
+Z Z
+[ [
+_ _
+a a
+b b
+c c
+d d
+e e
+f f
+g g
+h h
+i i
+j j
+k k
+l l
+m m
+n n
+o o
+p p
+q q
+r r
+s s
+t t
+u u
+v v
+w w
+x x
+y y
+z z
+{ {
+| |
+} }
+§ §
+« «
+° °
+² ²
+º º
+» »
+É É
+Ü Ü
+à à
+â â
+ç ç
+è è
+é é
+ê ê
+ë ë
+î î
+ï ï
+ô ô
+ù ù
+û û
+ü ü
+Œ Œ
+œ œ
+’ ’
+… …
+<unk> <unk>
+<space> <space>

pylaia/belfort/syms.txt

@@ -0,0 +1,112 @@
+<ctc> 0
+" 1
+% 2
+& 3
+' 4
+( 5
+) 6
+* 7
+, 8
+- 9
+. 10
+/ 11
+0 12
+1 13
+2 14
+3 15
+4 16
+5 17
+6 18
+7 19
+8 20
+9 21
+: 22
+; 23
+< 24
+= 25
+> 26
+? 27
+A 28
+B 29
+C 30
+D 31
+E 32
+F 33
+G 34
+H 35
+I 36
+J 37
+K 38
+L 39
+M 40
+N 41
+O 42
+P 43
+Q 44
+R 45
+S 46
+T 47
+U 48
+V 49
+W 50
+X 51
+Y 52
+Z 53
+[ 54
+_ 55
+a 56
+b 57
+c 58
+d 59
+e 60
+f 61
+g 62
+h 63
+i 64
+j 65
+k 66
+l 67
+m 68
+n 69
+o 70
+p 71
+q 72
+r 73
+s 74
+t 75
+u 76
+v 77
+w 78
+x 79
+y 80
+z 81
+{ 82
+| 83
+} 84
+§ 85
+« 86
+° 87
+² 88
+º 89
+» 90
+É 91
+Ü 92
+à 93
+â 94
+ç 95
+è 96
+é 97
+ê 98
+ë 99
+î 100
+ï 101
+ô 102
+ù 103
+û 104
+ü 105
+Π106
+œ 107
+’ 108
+… 109
+<unk> 110
+<space> 111

pylaia/belfort/tokens.txt

@@ -0,0 +1,112 @@
+<ctc>
+"
+%
+&
+'
+(
+)
+*
+,
+-
+.
+/
+0
+1
+2
+3
+4
+5
+6
+7
+8
+9
+:
+;
+<
+=
+>
+?
+A
+B
+C
+D
+E
+F
+G
+H
+I
+J
+K
+L
+M
+N
+O
+P
+Q
+R
+S
+T
+U
+V
+W
+X
+Y
+Z
+[
+_
+a
+b
+c
+d
+e
+f
+g
+h
+i
+j
+k
+l
+m
+n
+o
+p
+q
+r
+s
+t
+u
+v
+w
+x
+y
+z
+{
+|
+}
+§
+«
+°
+²
+º
+»
+É
+Ü
+à
+â
+ç
+è
+é
+ê
+ë
+î
+ï
+ô
+ù
+û
+ü
+Œ
+œ
+’
+…
+<unk>
+<space>