PaddleSpeech/examples/iwslt2012/punc0/README.md

Punctuation Restoration with IWLST2012-Zh

Get Started

Data Preprocessing

./run.sh --stage 0 --stop-stage 0

Model Training

./run.sh --stage 1 --stop-stage 1

Testing

./run.sh --stage 2 --stop-stage 2

Punctuation Restoration

./run.sh --stage 3 --stop-stage 3

Pretrained Model

The pretrained model can be downloaded here:

ernie_linear_p3_iwslt2012_zh_ckpt_0.1.1.zip

ernie-3.0-base.tar.gz

ernie-3.0-medium.tar.gz

ernie-3.0-micro.tar.gz

ernie-mini.tar.gz

ernie-nano.tar.gz

ernie-tiny.tar.gz

Test Result

• Ernie 1.0

  	COMMA	PERIOD	QUESTION	OVERALL
  Precision	0.510955	0.526462	0.820755	0.619391
  Recall	0.517433	0.564179	0.861386	0.647666
  F1	0.514173	0.544669	0.840580	0.633141

• Ernie-tiny

  	COMMA	PERIOD	QUESTION	OVERALL
  Precision	0.733177	0.721448	0.754717	0.736447
  Recall	0.380740	0.524646	0.733945	0.546443
  F1	0.501204	0.607506	0.744186	0.617632

• Ernie-3.0-base-zh

  	COMMA	PERIOD	QUESTION	OVERALL
  Precision	0.805947	0.764160	0.858491	0.809532
  Recall	0.399070	0.567978	0.850467	0.605838
  F1	0.533817	0.651623	0.854460	0.679967

• Ernie-3.0-medium-zh

  	COMMA	PERIOD	QUESTION	OVERALL
  Precision	0.730829	0.699164	0.707547	0.712514
  Recall	0.388196	0.533286	0.797872	0.573118
  F1	0.507058	0.605062	0.750000	0.620707

• Ernie-3.0-mini-zh

  	COMMA	PERIOD	QUESTION	OVERALL
  Precision	0.757433	0.708449	0.707547	0.724477
  Recall	0.355752	0.506977	0.735294	0.532674
  F1	0.484121	0.591015	0.721154	0.598763

• Ernie-3.0-micro-zh

  	COMMA	PERIOD	QUESTION	OVERALL
  Precision	0.733959	0.679666	0.726415	0.713347
  Recall	0.332742	0.483487	0.712963	0.509731
  F1	0.457896	0.565033	0.719626	0.580852

• Ernie-3.0-nano-zh

  	COMMA	PERIOD	QUESTION	OVERALL
  Precision	0.693271	0.682451	0.754717	0.710146
  Recall	0.327784	0.491968	0.666667	0.495473
  F1	0.445114	0.571762	0.707965	0.574947