# Chinese Text Frontend Example
Here's an example for Chinese text frontend, including g2p and text normalization.
## G2P
For g2p, we use BZNSYP's phone label as the ground truth and we delete silence tokens in labels and predicted phones.

You should Download BZNSYP from it's [Official Website](https://test.data-baker.com/data/index/source) and extract it. Assume the path to the dataset is `~/datasets/BZNSYP`.

We use `WER` as evaluation criterion.
## Text Normalization
For text normalization, the test data is  `data/textnorm_test_cases.txt`, we use `|` as the separator of raw_data and normed_data.

We use `CER` as evaluation criterion.
## Start
If you want to use sclite to get more detail information of WER, you should run the command below to make sclite first.
```bash
./make_sclite.sh
```
Run the command below to get the results of test.
```bash
./run.sh
```
The `avg WER` of g2p is: 0.027495061517943988
```text

                     SYSTEM SUMMARY PERCENTAGES by SPEAKER  

   ,------------------------------------------------------------------------.
   |                           ./exp/g2p/text.g2p                           |
   |------------------------------------------------------------------------|
   | SPKR | # Snt    # Wrd  |  Corr      Sub     Del    Ins    Err    S.Err |
   |------+-----------------+-----------------------------------------------|
   | bak  |  9996   299181  | 290969    8198      14     14   8226    5249  |
   |========================================================================|
   | Sum  |  9996   299181  | 290969    8198      14     14   8226    5249  |
   |========================================================================|
   | Mean |9996.0  299181.0 |290969.0  8198.0   14.0   14.0  8226.0  5249.0 |
   | S.D. |  0.0      0.0   |   0.0      0.0     0.0    0.0    0.0     0.0  |
   |Median|9996.0  299181.0 |290969.0  8198.0   14.0   14.0  8226.0  5249.0 |
   `------------------------------------------------------------------------'

                     SYSTEM SUMMARY PERCENTAGES by SPEAKER  

     ,--------------------------------------------------------------------.
     |                         ./exp/g2p/text.g2p                         |
     |--------------------------------------------------------------------|
     | SPKR   | # Snt    # Wrd  | Corr    Sub    Del    Ins    Err  S.Err |
     |--------+-----------------+-----------------------------------------|
     | bak    |  9996   299181  | 97.3    2.7    0.0    0.0    2.7   52.5 |
     |====================================================================|
     | Sum/Avg|  9996   299181  | 97.3    2.7    0.0    0.0    2.7   52.5 |
     |====================================================================|
     |  Mean  |9996.0  299181.0 | 97.3    2.7    0.0    0.0    2.7   52.5 |
     |  S.D.  |  0.0      0.0   |  0.0    0.0    0.0    0.0    0.0    0.0 |
     | Median |9996.0  299181.0 | 97.3    2.7    0.0    0.0    2.7   52.5 |
     `--------------------------------------------------------------------'
```

The `avg CER` of text normalization is: 0.006388318503308237
```text

                     SYSTEM SUMMARY PERCENTAGES by SPEAKER  

       ,----------------------------------------------------------------.
       |                     ./exp/textnorm/text.tn                     |
       |----------------------------------------------------------------|
       | SPKR | # Snt  # Wrd | Corr     Sub    Del    Ins    Err  S.Err |
       |------+--------------+------------------------------------------|
       | utt  |  125    2254 | 2241       2     11      2     15      4 |
       |================================================================|
       | Sum  |  125    2254 | 2241       2     11      2     15      4 |
       |================================================================|
       | Mean |125.0  2254.0 |2241.0    2.0   11.0    2.0   15.0    4.0 |
       | S.D. |  0.0    0.0  |  0.0     0.0    0.0    0.0    0.0    0.0 |
       |Median|125.0  2254.0 |2241.0    2.0   11.0    2.0   15.0    4.0 |
       `----------------------------------------------------------------'

                     SYSTEM SUMMARY PERCENTAGES by SPEAKER  

      ,-----------------------------------------------------------------.
      |                     ./exp/textnorm/text.tn                      |
      |-----------------------------------------------------------------|
      | SPKR   | # Snt  # Wrd | Corr    Sub    Del    Ins    Err  S.Err |
      |--------+--------------+-----------------------------------------|
      | utt    |  125    2254 | 99.4    0.1    0.5    0.1    0.7    3.2 |
      |=================================================================|
      | Sum/Avg|  125    2254 | 99.4    0.1    0.5    0.1    0.7    3.2 |
      |=================================================================|
      |  Mean  |125.0  2254.0 | 99.4    0.1    0.5    0.1    0.7    3.2 |
      |  S.D.  |  0.0    0.0  |  0.0    0.0    0.0    0.0    0.0    0.0 |
      | Median |125.0  2254.0 | 99.4    0.1    0.5    0.1    0.7    3.2 |
      `-----------------------------------------------------------------'
```