This Library is mainly for those implementing a Terminal Emulator, or programs that carefully produce output to mimick or to be interpreted by an emulator.
Problem Statement: The printible length of most strings are equal to the number of cells they occupy on the screen. However, there are categories of characters that occupy 2 cells (full-wide), and others that occupy 0 cells (zero-width).
Solution: POSIX.1-2001 and POSIX.1-2008 conforming systems provide wcwidth(3) and wcswidth(3) C functions of which this python module's functions precisely copy. These functions return the number of cells a unicode string is expected to occupy.
The stable version of this package is maintained on pypi, install or upgrade using pip:
pip install --upgrade wcwidth
Problem: given the following phrase (Japanese),
>>> text = u'コンニチハ'
Python incorrectly uses the string length of 5 codepoints rather than the printible length of 10 cells, so that when using the rjust function, the output length is wrong:
>>> print(len('コンニチハ')) 5 >>> print('コンニチハ'.rjust(11, '_')) ______コンニチハ
By defining our own "rjust" function that uses wcwidth, we can correct this:
>>> def wc_rjust(text, length, padding=' '): ... from wcwidth import wcswidth ... return padding * max(0, (length - wcswidth(text))) + text ...
Our Solution uses wcswidth to determine the string length correctly:
>>> from wcwidth import wcswidth >>> print(wcswidth('コンニチハ')) 10 >>> print(wc_rjust('コンニチハ', 11, '_')) _コンニチハ
This library is used in:
- asciimatics: Package to help people create full-screen text UIs.
- blessed: a simplified wrapper around curses.
- curtsies: Curses wrapper with a display based on compositing 2d arrays of text.
- ftfy: Fixes mojibake and other glitches in Unicode text, after the fact.
- pyte: a Simple VTXXX-compatible linux terminal emulator.
- python-prompt-toolkit: a Powerful interactive command line building library.
- termtosvg: Terminal recorder that renders sessions as SVG animations.
Use function wcwidth()
to determine the length of a single unicode
character, and wcswidth()
to determine the length of a several, or a
string of unicode characters.
Briefly, return values of function wcwidth()
are:
-1
- Indeterminate (not printable).
0
- Does not advance the cursor, such as NULL or Combining.
2
- Characters of category East Asian Wide (W) or East Asian Full-width (F) which are displayed using two terminal cells.
1
- All others.
Function wcswidth()
simply returns the sum of all values for each character
along a string, or -1
when it occurs anywhere along a string.
More documentation is available using pydoc:
$ pydoc wcwidth
This library attempts to determine the printable width by a fictional terminal, using the very latest Unicode specification, though some work is in progress for the ability to select a version, there is no standards conforming ability to discern what the target emulator software, version, of level of support is. Results may vary!
A crude method of determining the level of unicode support by the target emulator may be performed using the VT100 Query Cursor Position sequence.
The libc version of wcwidth(3) is often several unicode releases behind, and therefor several levels of support lower than this python library.
Install wcwidth in editable mode:
pip install -e.
Install developer requirements:
pip install -r requirements-develop.txt
Execute unit tests using tox:
tox
Use the interactive browser:
python bin/wcwidth-browser.py
This library aims to be forward-looking, portable, and most correct. The most current release of this API is based on the Unicode Standard release files:
DerivedGeneralCategory-13.0.0.txt
- Date: 2019-10-21, 14:30:32 GMT © 2019 Unicode®, Inc.
EastAsianWidth-13.0.0.txt
- Date: 2020-01-21, 18:14:00 GMT [KW, LI] © 2020 Unicode®, Inc.
The command python setup.py update
will fetch the following resources:
- http://www.unicode.org/Public/UNIDATA/EastAsianWidth.txt
- http://www.unicode.org/Public/UNIDATA/extracted/DerivedGeneralCategory.txt
And generates the table files:
- 0.1.9 2020-03-22
- Performance optimization by Avram Lubkin, PR #35.
- Updated tables to Unicode Specification 13.0.0.
- 0.1.8 2020-01-01
- Updated tables to Unicode Specification 12.0.0. (PR #30).
- 0.1.7 2016-07-01
- Updated tables to Unicode Specification 9.0.0. (PR #18).
- 0.1.6 2016-01-08 Production/Stable
LICENSE
file now included with distribution.
- 0.1.5 2015-09-13 Alpha
- Bugfix: Resolution of "combining character width" issue, most especially those that previously returned -1 now often (correctly) return 0. resolved by Philip Craig via PR #11.
- Deprecated:
The module path
wcwidth.table_comb
is no longer available, it has been superseded by module pathwcwidth.table_zero
.
- 0.1.4 2014-11-20 Pre-Alpha
- Feature:
wcswidth()
now determines printable length for (most) combining characters. The developer's tool bin/wcwidth-browser.py is improved to display combining characters when provided the--combining
option (Thomas Ballinger and Leta Montopoli PR #5). - Feature: added static analysis (prospector) to testing framework.
- Feature:
- 0.1.3 2014-10-29 Pre-Alpha
- Bugfix: 2nd parameter of wcswidth was not honored. (Thomas Ballinger, PR #4).
- 0.1.2 2014-10-28 Pre-Alpha
- Updated tables to Unicode Specification 7.0.0. (Thomas Ballinger, PR #3).
- 0.1.1 2014-05-14 Pre-Alpha
- Initial release to pypi, Based on Unicode Specification 6.3.0
This code was originally derived directly from C code of the same name, whose latest version is available at http://www.cl.cam.ac.uk/~mgk25/ucs/wcwidth.c:
* Markus Kuhn -- 2007-05-26 (Unicode 5.0) * * Permission to use, copy, modify, and distribute this software * for any purpose and without fee is hereby granted. The author * disclaims all warranties with regard to this software.