|
|
|
|
# Punctuation Restoration with IWLST2012-Zh
|
|
|
|
|
|
|
|
|
|
## Get Started
|
|
|
|
|
### Data Preprocessing
|
|
|
|
|
```bash
|
|
|
|
|
./run.sh --stage 0 --stop-stage 0
|
|
|
|
|
```
|
|
|
|
|
### Model Training
|
|
|
|
|
```bash
|
|
|
|
|
./run.sh --stage 1 --stop-stage 1
|
|
|
|
|
```
|
|
|
|
|
### Testing
|
|
|
|
|
```bash
|
|
|
|
|
./run.sh --stage 2 --stop-stage 2
|
|
|
|
|
```
|
|
|
|
|
### Punctuation Restoration
|
|
|
|
|
```bash
|
|
|
|
|
./run.sh --stage 3 --stop-stage 3
|
|
|
|
|
```
|
|
|
|
|
## Pretrained Model
|
|
|
|
|
The pretrained model can be downloaded here:
|
|
|
|
|
|
|
|
|
|
[ernie_linear_p3_iwslt2012_zh_ckpt_0.1.1.zip](https://paddlespeech.bj.bcebos.com/text/ernie_linear_p3_iwslt2012_zh_ckpt_0.1.1.zip)
|
|
|
|
|
|
|
|
|
|
[ernie-3.0-base.tar.gz](https://paddlespeech.bj.bcebos.com/punc_restore/ernie-3.0-base.tar.gz)
|
|
|
|
|
|
|
|
|
|
[ernie-3.0-medium.tar.gz](https://paddlespeech.bj.bcebos.com/punc_restore/ernie-3.0-medium.tar.gz)
|
|
|
|
|
|
|
|
|
|
[ernie-3.0-micro.tar.gz](https://paddlespeech.bj.bcebos.com/punc_restore/ernie-3.0-micro.tar.gz)
|
|
|
|
|
|
|
|
|
|
[ernie-mini.tar.gz](https://paddlespeech.bj.bcebos.com/punc_restore/ernie-mini.tar.gz)
|
|
|
|
|
|
|
|
|
|
[ernie-nano.tar.gz](https://paddlespeech.bj.bcebos.com/punc_restore/ernie-nano.tar.gz)
|
|
|
|
|
|
|
|
|
|
[ernie-tiny.tar.gz](https://paddlespeech.bj.bcebos.com/punc_restore/ernie-tiny.tar.gz)
|
|
|
|
|
|
|
|
|
|
### Test Result
|
|
|
|
|
- Ernie 1.0
|
|
|
|
|
| |COMMA | PERIOD | QUESTION | OVERALL|
|
|
|
|
|
|:-----:|:-----:|:-----:|:-----:|:-----:|
|
|
|
|
|
|Precision |0.510955 |0.526462 |0.820755 |0.619391|
|
|
|
|
|
|Recall |0.517433 |0.564179 |0.861386 |0.647666|
|
|
|
|
|
|F1 |0.514173 |0.544669 |0.840580 |0.633141|
|
|
|
|
|
- Ernie-tiny
|
|
|
|
|
| |COMMA | PERIOD | QUESTION | OVERALL|
|
|
|
|
|
|:-----:|:-----:|:-----:|:-----:|:-----:|
|
|
|
|
|
|Precision |0.733177 |0.721448 |0.754717 |0.736447|
|
|
|
|
|
|Recall |0.380740 |0.524646 |0.733945 |0.546443|
|
|
|
|
|
|F1 |0.501204 |0.607506 |0.744186 |0.617632|
|
|
|
|
|
- Ernie-3.0-base-zh
|
|
|
|
|
| |COMMA | PERIOD | QUESTION | OVERALL|
|
|
|
|
|
|:-----:|:-----:|:-----:|:-----:|:-----:|
|
|
|
|
|
|Precision |0.805947 |0.764160 |0.858491 |0.809532|
|
|
|
|
|
|Recall |0.399070 |0.567978 |0.850467 |0.605838|
|
|
|
|
|
|F1 |0.533817 |0.651623 |0.854460 |0.679967|
|
|
|
|
|
- Ernie-3.0-medium-zh
|
|
|
|
|
| |COMMA | PERIOD | QUESTION | OVERALL|
|
|
|
|
|
|:-----:|:-----:|:-----:|:-----:|:-----:|
|
|
|
|
|
|Precision |0.730829 |0.699164 |0.707547 |0.712514|
|
|
|
|
|
|Recall |0.388196 |0.533286 |0.797872 |0.573118|
|
|
|
|
|
|F1 |0.507058 |0.605062 |0.750000 |0.620707|
|
|
|
|
|
- Ernie-3.0-mini-zh
|
|
|
|
|
| |COMMA | PERIOD | QUESTION | OVERALL|
|
|
|
|
|
|:-----:|:-----:|:-----:|:-----:|:-----:|
|
|
|
|
|
|Precision |0.757433 |0.708449 |0.707547 |0.724477|
|
|
|
|
|
|Recall |0.355752 |0.506977 |0.735294 |0.532674|
|
|
|
|
|
|F1 |0.484121 |0.591015 |0.721154 |0.598763|
|
|
|
|
|
- Ernie-3.0-micro-zh
|
|
|
|
|
| |COMMA | PERIOD | QUESTION | OVERALL|
|
|
|
|
|
|:-----:|:-----:|:-----:|:-----:|:-----:|
|
|
|
|
|
|Precision |0.733959 |0.679666 |0.726415 |0.713347|
|
|
|
|
|
|Recall |0.332742 |0.483487 |0.712963 |0.509731|
|
|
|
|
|
|F1 |0.457896 |0.565033 |0.719626 |0.580852|
|
|
|
|
|
- Ernie-3.0-nano-zh
|
|
|
|
|
| |COMMA | PERIOD | QUESTION | OVERALL|
|
|
|
|
|
|:-----:|:-----:|:-----:|:-----:|:-----:|
|
|
|
|
|
|Precision |0.693271 |0.682451 |0.754717 |0.710146|
|
|
|
|
|
|Recall |0.327784 |0.491968 |0.666667 |0.495473|
|
|
|
|
|
|F1 |0.445114 |0.571762 |0.707965 |0.574947|
|
