Commit e68e0303 authored by Dan Povey's avatar Dan Povey
Browse files

Adding new results in paper.tex at bottom

git-svn-id: https://svn.code.sf.net/p/kaldi/code/trunk@528 5e6a8d80-dfce-4ca6-a32a-6e07a63d50c8
parent b4fe79a9
......@@ -565,18 +565,20 @@ have a wide range over files: for example beam=15.0, lat_beam=9.0:
ranging from 1.5 to 167
running on blades, using 64bit version with double float and lattice-simple-decoder
realtime factor running on pcburget, 22989 frames (decode3.sh)
decoding with bigram LM:
decoding WER: 11.51%
ins/del/sub: 118/57/474
utterances wrong: 227
experiment: changing lattice beam:
1) experiment: changing lattice beam:
(lattices 443c040w and 444c040i have been reduced from 10.0 to 8.1 because of determinization memory problems)
decoding beam: 15.0 15.0 15.0 15.0 15.0 15.0 15.0 15.0 15.0 15.0 15.0
lattice beam: 10.0 9.0 8.0 7.0 6.0 5.0 4.0 3.0 2.0 1.0 0.0
avg. link density: 23.56 16.0 10.82 7.26 4.91 3.38 2.38 1.77 1.4 1.17 1.0
realtime factor: 3.61 3.17 2.59 2.41 2.34 2.98 ? 3.21 ? 1.89 3.12 ? 1.78 --
realtime factor: 2.317 2.114 2.006 1.941 1.900 1.855 1.835 1.797 1.769 1.737 ---
oracle WER: 2.62% 2.78% 2.85% 3.08% 3.35% 3.86% 4.52% 5.46% 6.98% 8.62% 11.51%
ins/del/sub: 22/0/126 25/0/132 29/0/132 31/1/142 38/5/146 51/11/156 57/15/183 64/20/224 76/32/286 89/40/357 118/57/474
......@@ -585,3 +587,65 @@ oracle utt wrong: 93 96 96 101 105 11
rescoring WER(15): 9.59% 9.59% 9.59% 9.59% 9.56% 9.59% 9.61% 9.70% 9.80% 10.21% 11.51%
ins/del/sub: 117/36/388 116/36/387 117/35/389 115/35/392 111/38/398 108/39/406 113/42/421 118/57/474
rescore utt wrong: 205 204 204 204 202 205 212 227
2) experiment: changing decoding beam:
decoding beam: 16.0 15.0 14.0 13.0 12.0 11.0 10.0 9.0 (1 utt partial) 8.0 (2 partial)
lattice beam: 7.0 7.0 7.0 7.0 7.0 7.0 7.0 7.0 7.0
avg. link density: 7.4 7.26 7.04 6.74 6.33 5.71 4.9 4.08 3.09
realtime factor: 2.646 1.941 1.428 1.052 0.775 0.555 0.388 0.267 0.177
decoding WER: 11.51% 11.51% 11.54% 11.58% 11.75% 11.93% 12.32% 13.47% 15.41%
ins/del/sub: 118/57/474 118/57/474 118/57/476 119/56/478 124/57/482 131/58/484 140/58/497 164/56/540 197/61/611
utterances wrong: 227 227 227 227 228 228 229 236 246
oracle WER: 3.03% 3.08% 3.24% 3.37% 3.56% 4.02% 4.72% 6.45% 9.18%
ins/del/sub: 30/1/140 31/1/142 35/2/146 38/2/150 42/3/156 54/6/167 55/7/204 73/13/278 118/22/378
oracle utt wrong: 100 101 105 108 112 119 128 153 184
rescoring WER(15): 9.59% 9.59% 9.73% 9.80% 10.05% 10.26% 10.55% 11.82% (1 failed) 13.79% (2 failed)
ins/del/sub: 116/36/389 117/36/388 115/37/397 117/38/398 123/39/405 132/39/408 141/37/417 --- ---
rescore utt wrong: 205 205 206 206 207 209 208 220 235
3) bigram vs. trigram decoding:
decoding lm: bigram trigram,pruned bigram trigram,pruned bigram (just to see whether wider beams still help)
decoding beam: 15.0 15.0 13.0 13.0 16.0
lattice beam: 9.0 9.0 9.2 9.2 10.0 (already quite big for lattice-oracle - more than 6GB RAM used)
avg. link density: 16.0 14.98 14.86 13.58 24.84
rt factor(old): 3.17 3.26 1.75 1.77 4.54
decoding WER: 11.51% 10.57% 11.58% 10.80% 11.51%
ins/del/sub: 118/57/474 116/44/436 119/56/478 120/47/442 118/57/474
utterances wrong: 227 214 227 215 227
oracle WER: 2.78% 2.84% 3.05% 3.19% 2.61%
ins/del/sub: 25/0/132 30/0/130 30/2/140 33/1/146 21/0/126
oracle utt wrong: 96 96 103 107 92
rescoring WER(15): 9.59% 9.64% 9.80% 9.87% 9.59%
ins/del/sub: 117/36/388 115/34/395 117/38/398 119/38/400 116/36/389
rescore utt wrong: 205 205 206 209 205
4) lattice vs. n-best lists
decoding with trigram, pruned
decoding: lattice n-best n-best/beam
decoding beam: 13.0 13.0 13.0
lattice beam: 9.2 -- 9.35 [6.0-12.5] (few outliers below/above)
N best: -- <=20 206.96 [2-1600] (according to beam range and utterance)
max tokens: 15000 150k,700k 150000-770000
avg. link density: 13.58 14? 231.05
realtime factor: 1.77 12 crap
decoding WER: 10.80% 12.78% (why?) 14.96% (probably too low max tokens)
ins/del/sub: 120/47/442
utterances wrong: 215
oracle WER: 3.19%
ins/del/sub: 33/1/146
oracle utt wrong: 107
rescoring WER(15): 9.87%
ins/del/sub: 119/38/400
rescore utt wrong: 209
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment