Phonemizer -- foʊnmaɪzɚ
-
The phonemizer allows simple phonemization of words and texts in many languages.
-
Provides both the
phonemize
command-line tool and the Python functionphonemizer.phonemize
. -
It is using four backends: espeak, espeak-mbrola, festival and segments.
-
espeak supports a lot of languages and IPA (International Phonetic Alphabet) output.
-
espeak-mbrola uses the SAMPA phonetic alphabet instead of IPA but does not preserve word boundaries.
-
festival currently supports only American English. It uses a custom phoneset, but it allows tokenization at the syllable level.
-
segments is a Unicode tokenizer that build a phonemization from a grapheme to phoneme mapping provided as a file by the user.
-
Installation
You need python>=3.6. If you really need to use python2, use an older version of the phonemizer.
Dependencies
-
You need to install festival, espeak-ng and mbrola on your system. On Debian/Ubuntu simply run:
$ sudo apt-get install festival espeak-ng mbrola
-
Alternatively you may want to use espeak instead of espeak-ng, see here for installation instructions.
-
When using the espeak-mbrola backend, additional mbrola voices must be installed (see here). On Debian/Ubuntu, list the installable voices with
apt search mbrola
.
Phonemizer
-
The simplest way is using pip:
$ pip install phonemizer
-
OR install it from sources with:
$ git clone https://github.com/bootphon/phonemizer $ cd phonemizer $ python setup.py build $ [sudo] python setup.py install
If you experiment an error such as
ImportError: No module named setuptools
during installation, refeer to issue 11.
Docker image
Alternatively you can run the phonemizer within docker, using the
provided Dockerfile
. To build the docker image, have a:
$ git clone https://github.com/bootphon/phonemizer
$ cd phonemizer
$ sudo docker build -t phonemizer .
Then run an interactive session with:
$ sudo docker run -it phonemizer /bin/bash
Command-line examples
For a complete list of available options, have a:
$ phonemize --help
See the installed backends with the --version
option:
$ phonemize --version
phonemizer-2.2
available backends: espeak-ng-1.49.3, espeak-mbrola, festival-2.5.0, segments-2.0.1
Input/output exemples
-
from stdin to stdout:
$ echo "hello world" | phonemize həloʊ wɜːld
-
from file to stdout
$ echo "hello world" > hello.txt $ phonemize hello.txt həloʊ wɜːld
-
from file to file
$ phonemize hello.txt -o hello.phon --strip $ cat hello.phon həloʊ wɜːld
Backends
-
The default is to use espeak us-english:
$ echo "hello world" | phonemize həloʊ wɜːld $ echo "hello world" | phonemize -l en-us -b espeak həloʊ wɜːld
-
Use festival US English instead
$ echo "hello world" | phonemize -l en-us -b festival hhaxlow werld
-
In French, using espeak and espeak-mbrola, with custom token separators (see below). espeak-mbrola does not support words separation.
$ echo "bonjour le monde" | phonemize -b espeak -l fr-fr -p ' ' -w '/w ' b ɔ̃ ʒ u ʁ /w l ə /w m ɔ̃ d /w $ echo "bonjour le monde" | phonemize -b espeak-mbrola -l mb-fr1 -p ' ' -w '/w ' b o~ Z u R l @ m o~ d
-
In Japanese, using segments
$ echo 'konnichiwa' | phonemize -b segments -l japanese konnitʃiwa $ echo 'konnichiwa' | phonemize -b segments -l ./phonemizer/share/japanese.g2p konnitʃiwa
Supported languages
The exhaustive list of supported languages is available with the command
phonemize --list-languages [--backend <backend>]
.
-
Languages supported by espeak are available here.
-
Languages supported by espeak-mbrola are available here. Please note that the mbrola voices are not bundled with the phonemizer and must be installed separately.
-
Languages supported by festival are:
en-us -> english-us
-
Languages supported by the segments backend are:
chintang -> ./phonemizer/share/segments/chintang.g2p cree -> ./phonemizer/share/segments/cree.g2p inuktitut -> ./phonemizer/share/segments/inuktitut.g2p japanese -> ./phonemizer/share/segments/japanese.g2p sesotho -> ./phonemizer/share/segments/sesotho.g2p yucatec -> ./phonemizer/share/segments/yucatec.g2p
Instead of a language you can also provide a file specifying a grapheme to phone mapping (see the files above for examples).
Token separators
You can specify separators for phones, syllables (festival only) and words (excepted espeak-mbrola).
$ echo "hello world" | phonemize -b festival -w ' ' -p ''
hhaxlow werld
$ echo "hello world" | phonemize -b festival -p ' ' -w ''
hh ax l ow w er l d
$ echo "hello world" | phonemize -b festival -p '-' -s '|'
hh-ax-l-|ow-| w-er-l-d-|
$ echo "hello world" | phonemize -b festival -p '-' -s '|' --strip
hh-ax-l|ow w-er-l-d
$ echo "hello world" | phonemize -b festival -p ' ' -s ';esyll ' -w ';eword '
hh ax l ;esyll ow ;esyll ;eword w er l d ;esyll ;eword
You cannot specify the same separator for several tokens (for instance a space for both phones and words):
$ echo "hello world" | phonemize -b festival -p ' ' -w ' '
fatal error: illegal separator with word=" ", syllable="" and phone=" ",
must be all differents if not empty
Punctuation
By default the punctuation is removed in the phonemized output. You can preserve
it using the --preserve-punctuation
option (not supported by the
espeak-mbrola backend):
$ echo "hello, world!" | phonemize --strip
həloʊ wɜːld
$ echo "hello, world!" | phonemize --preserve-punctuation --strip
həloʊ, wɜːld!
Espeak specific options
-
The espeak backend can output the stresses on phones:
$ echo "hello world" | phonemize -l en-us -b espeak --with-stress həlˈoʊ wˈɜːld
-
The espeak backend can switch languages during phonemization (below from French to English), use the
--language-switch
option to deal with it:$ echo "j'aime le football" | phonemize -l fr-fr -b espeak --language-switch keep-flags [WARNING] fount 1 utterances containing language switches on lines 1 [WARNING] extra phones may appear in the "fr-fr" phoneset [WARNING] language switch flags have been kept (applying "keep-flags" policy) ʒɛm lə- (en)fʊtbɔːl(fr) $ echo "j'aime le football" | phonemize -l fr-fr -b espeak --language-switch remove-flags [WARNING] fount 1 utterances containing language switches on lines 1 [WARNING] extra phones may appear in the "fr-fr" phoneset [WARNING] language switch flags have been removed (applying "remove-flags" policy) ʒɛm lə- fʊtbɔːl $ echo "j'aime le football" | phonemize -l fr-fr -b espeak --language-switch remove-utterance [WARNING] removed 1 utterances containing language switches (applying "remove-utterance" policy)
Licence
Copyright 2015-2020 Mathieu Bernard
This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/.