revise the install.md, setup.py and makefile, rm the setup.sh

pull/1063/head
huangyuxin 3 years ago
parent 989aec4413
commit 719d23c07a

@ -1,53 +1,101 @@
# Installation
To avoid the trouble of environment setup, [running in Docker container](#running-in-docker-container) is highly recommended. Otherwise follow the guidelines below to install the dependencies manually.
There are 3 ways to use the repository. According to the degree of difficulty, the 3 ways can be divided into Easy, Medium and Hard.
## Easy: Get the Basic Funcition Without Your Own Mechine
If you are in touch with PaddleSpeech for the first time and want to experience it easily without your own mechine. We recommand you to go to aistudio to experience the PaddleSpeech project. There is a step-by-step tutorial for PaddleSpeech and you can use the basic function of PaddleSpeech with a free machine.
## Prerequisites for Medium and Hard
## Prerequisites
- Python >= 3.7
- PaddlePaddle latest version (please refer to the [Installation Guide](https://www.paddlepaddle.org.cn/documentation/docs/en/beginners_guide/index_en.html))
- Only Linux is supported
- Hip: Do not use command `sh` instead of command `bash`
## Medium: Get the Basic Funciton on Your Mechine
## Simple Setup
If you want to install the paddlespeech on your own mechine. There are 3 steps you need to do.
For user who working on `Ubuntu` with `root` privilege.
### Install PaddlePaddle
```bash
git clone https://github.com/PaddlePaddle/DeepSpeech.git
cd PaddleSpeech
python3 -m pip install paddlepaddle-gpu==2.2.0
```
If you want to use the basic function of the repo, you can use:
### Install the Conda
The first setup is installing the conda. Conda is environment management system. You can go to [minicoda](https://docs.conda.io/en/latest/miniconda.html) to select a version (py>=3.7) and install it by yourself or you can use the scripts below:
```bash
pip install .
# download the miniconda
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
# install the miniconda
bash Miniconda3-latest-Linux-x86_64.sh -b
# conda init
$HOME/miniconda3/bin/conda init
# activate the conda
bash
```
If you want to do the development, you can use:
Then you can create an conda virtual environment using the script:
```bash
conda create -n py37 python=3.7
```
pip install -e .[develop]
Activate the conda virtual environment:
```bash
conda activate py37
```
For user who only needs the basic function of paddlespeech, using conda to do installing is recommended.
You can go to [minicoda](https://docs.conda.io/en/latest/miniconda.html) to select a version and install it by yourself, or you can use the scripts below to install the last miniconda version.
Intall the conda dependencies
```bash
pushd tools
bash extras/install_miniconda.sh
popd
bash
conda install -c conda-forge sox libsndfile swig bzip2 gcc_linux-64=8.4.0 gxx_linux-64=8.4.0 --yes
```
After installing the conda, run the setup.sh to complete the installing process.
### Install the PaddleSpeech Using PiP
To Install the PaddleSpeech, there are two methods. You can use the script below:
```bash
bash setup.sh
pip install paddlespeech
```
If you install the paddlespeech by pip, you can use it to help you to build your own model. However, you can not use the ready-made examples in paddlespeech.
## Setup (Other Platform)
If you want to use the ready-made examples in paddlespeech, you need to clone the repository and install the paddlespeech package.
- Make sure these libraries or tools in [dependencies](./dependencies.md) installed. More information please see: `setup.py `and ` tools/Makefile`.
- The version of `swig` should >= 3.0
- we will do more to simplify the install process.
```bash
https://github.com/PaddlePaddle/PaddleSpeech.git
## Into the PaddleSpeech
cd PaddleSpeech
pip install .
```
## Hard: Get the Full Funciton on Your Mechine
### Prerequisites
- choice 1: working with `ubuntu` Docker Container.
or
## Running in Docker Container (optional)
- choice 2: working on `Ubuntu` with `root` privilege.
To avoid the trouble of environment setup, [running in Docker container](#running-in-docker-container) is highly recommended. Otherwise If you work on `Ubuntu` with `root` privilege, you can skip the next step.
### Choice 1: Running in Docker Container (Recommand)
Docker is an open source tool to build, ship, and run distributed applications in an isolated environment. A Docker image for this project has been provided in [hub.docker.com](https://hub.docker.com) with all the dependencies installed. This Docker image requires the support of NVIDIA GPU, so please make sure its availiability and the [nvidia-docker](https://github.com/NVIDIA/nvidia-docker) has been installed.
@ -55,22 +103,22 @@ Take several steps to launch the Docker image:
- Download the Docker image
For example, pull paddle 2.0.0 image:
For example, pull paddle 2.2.0 image:
```bash
nvidia-docker pull registry.baidubce.com/paddlepaddle/paddle:2.0.0-gpu-cuda10.1-cudnn7
nvidia-docker pull registry.baidubce.com/paddlepaddle/paddle:2.2.0-gpu-cuda10.2-cudnn7
```
- Clone this repository
```
git clone https://github.com/PaddlePaddle/DeepSpeech.git
git clone https://github.com/PaddlePaddle/PaddleSpeech.git
```
- Run the Docker image
```bash
sudo nvidia-docker run --rm -it -v $(pwd)/DeepSpeech:/DeepSpeech registry.baidubce.com/paddlepaddle/paddle:2.0.0-gpu-cuda10.1-cudnn7 /bin/bash
sudo nvidia-docker run --rm -it -v $(pwd)/PaddleSpeech:/PaddleSpeech registry.baidubce.com/paddlepaddle/paddle:2.2.0-gpu-cuda10.2-cudnn7 /bin/bash
```
Now you can execute training, inference and hyper-parameters tuning in the Docker container.
@ -78,12 +126,66 @@ Now you can execute training, inference and hyper-parameters tuning in the Docke
- Install PaddlePaddle
For example, for CUDA 10.1, CuDNN7.5 install paddle 2.0.0:
For example, for CUDA 10.2, CuDNN7.5 install paddle 2.2.0:
```bash
python3 -m pip install paddlepaddle-gpu==2.2.0
```
### Choice 2: Running in Ubuntu with Root Privilege
- Clone this repository
```
git clone https://github.com/PaddlePaddle/PaddleSpeech.git
```
Install paddle 2.2.0:
```bash
python3 -m pip install paddlepaddle-gpu==2.2.0
```
### Install the Conda
```bash
# download and install the miniconda
pushd tools
bash extras/install_miniconda.sh
popd
# use the "bash" command to make the conda environment works
bash
# create an conda virtual environment
conda create -n py37 python=3.7
# Activate the conda virtual environment:
conda activate py37
# Install the conda packags
conda install -c conda-forge sox libsndfile swig bzip2 gcc_linux-64=8.4.0 gxx_linux-64=8.4.0 --yes
```
### Get the Funcition for Developing PaddleSpeech
```bash
python3 -m pip install paddlepaddle-gpu==2.0.0
pip install .[develop]
```
- Install Deepspeech
### Install the Kaldi
Please see [Setup](#setup) section.
```bash
pushd tools
bash extras/install_openblas.sh
bash extras/install_kaldi.sh
popd
```
## Setup for Other Platform
- Make sure these libraries or tools in [dependencies](./dependencies.md) installed. More information please see: `setup.py `and ` tools/Makefile`.
- The version of `swig` should >= 3.0
- we will do more to simplify the install process.
- Install Paddlespeech

@ -124,25 +124,12 @@ def _post_install(install_lib_dir):
check_call("make")
print("tools install.")
# install autolog
tools_extrs_dir = HERE / 'tools/extras'
with pushd(tools_extrs_dir):
print(os.getcwd())
check_call("./install_autolog.sh")
print("autolog install.")
# ctcdecoder
ctcdecoder_dir = HERE / 'paddlespeech/s2t/decoders/ctcdecoder/swig'
with pushd(ctcdecoder_dir):
check_call("bash -e setup.sh")
print("ctcdecoder install.")
# install third_party
third_party_dir = HERE / 'third_party'
with pushd(third_party_dir):
check_call("bash -e install.sh")
print("third_party install.")
class DevelopCommand(develop):
def run(self):
develop.run(self)

@ -1,20 +0,0 @@
# Install conda dependencies
conda install -c conda-forge sox libsndfile swig bzip2 bottleneck gcc_linux-64=8.4.0 gxx_linux-64=8.4.0 --yes
# Install the python lib
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
# Install the auto_log
pushd tools/extras
bash install_autolog.sh
popd
# Install the ctcdecoder
pushd paddlespeech/s2t/decoders/ctcdecoder/swig
bash -e setup.sh
popd
# Install the python_speech_features
pushd third_party
bash -e install.sh
popd

@ -10,7 +10,7 @@ WGET ?= wget --no-check-certificate
.PHONY: all clean
all: virtualenv.done apt.done kenlm.done sox.done mfa.done sclite.done
all: apt.done kenlm.done mfa.done sclite.done
virtualenv.done:
test -d venv || virtualenv -p $(PYTHON) venv
@ -35,7 +35,7 @@ kenlm.done:
apt-get install -y gcc-5 g++-5 && update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-5 50 && update-alternatives --install /usr/bin/g++ g++ /usr/bin/g++-5 50
test -d kenlm || $(WGET) -O - https://kheafield.com/code/kenlm.tar.gz | tar xz
rm -rf kenlm/build && mkdir -p kenlm/build && cd kenlm/build && cmake .. && make -j4 && make install
cd kenlm && python setup.py install
cd kenlm && python3 setup.py install
touch kenlm.done
sox.done:

Loading…
Cancel
Save