fix detoken for char

pull/885/head
Hui Zhang 3 years ago
parent eef8847a82
commit 8ebd4245d7

@ -140,7 +140,7 @@ class TextFeaturizer():
Returns:
str: text string.
"""
tokens = tokens.replace(SPACE, " ")
tokens = [t.replace(SPACE, " ") for t in tokens ]
return "".join(tokens)
def word_tokenize(self, text):

@ -11,6 +11,7 @@
## Chunk Conformer
Need set `decoding.decoding_chunk_size=16` when decoding.
| Model | Params | Config | Augmentation| Test set | Decode method | Chunk Size & Left Chunks | Loss | WER |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
@ -18,10 +19,3 @@
| conformer | 47.06M | conf/chunk_conformer.yaml | spec_aug + shift | test | ctc_greedy_search | 16, -1 | - | 0.070806 |
| conformer | 47.06M | conf/chunk_conformer.yaml | spec_aug + shift | test | ctc_prefix_beam_search | 16, -1 | - | 0.070739 |
| conformer | 47.06M | conf/chunk_conformer.yaml | spec_aug + shift | test | attention_rescoring | 16, -1 | - | 0.059400 |
## Transformer
| Model | Params | Config | Augmentation| Test set | Decode method | Loss | WER |
| --- | --- | --- | --- | --- | --- | --- | ---|
| transformer | - | conf/transformer.yaml | spec_aug + shift | test | attention | - | - |

Loading…
Cancel
Save