Infinite recursion for `tesseract --list-langs` with conda-forge binary
jonashaag opened this issue · comments
Current Behavior
tesseract --list-langs goes into infinite loop on macOS if TESSDATA_PREFIX is empty.
macOS Instruments shows infinite recursion in addAvailableLanguages, and a LOT of stat64 calls (multiple 10k per second).
Expected Behavior
Should not go into infinite recursion
Suggested Fix
No response
tesseract -v
tesseract 5.3.4
leptonica-1.83.1
libgif 5.2.1 : libjpeg 8d (libjpeg-turbo 3.0.0) : libpng 1.6.43 : libtiff 4.6.0 : zlib 1.2.13 : libwebp 1.3.2 : libopenjp2 2.5.2
Found NEON
Operating System
macOS 14 Sonoma
Other Operating System
No response
uname -a
Darwin ... 23.4.0 Darwin Kernel Version 23.4.0: Fri Mar 15 00:12:41 PDT 2024; root:xnu-10063.101.17~1/RELEASE_ARM64_T8103 arm64
Compiler
From conda-forge
CPU
No response
Virtualization / Containers
None
Other Information
No response
Can you please exactly describe how to reproduce the problem?
How did you install tesseract?
I do not have macOS, but on Windows, I got this result:
> set TESSDATA_PREFIX=""
> tesseract --list-langs
List of available languages in """/" (0):
>set TESSDATA_PREFIX=
>tesseract --list-langs
List of available languages in "./" (1):
Downloads/dotslayerOn the Linux I got this result:
$ export TESSDATA_PREFIX=""
$ tesseract --list-langs
List of available languages in "/usr/share/tesseract-ocr/5/tessdata/" (2):
eng
osdOn macOS, your Linux commands reproduce the infinite loop
@jonashaag : how did you installed tesseract?
@stweil : Can you have a look at this?
I don't get an infinite loop on macOS (installed with Homebrew). An empty TESSDATA_PREFIX is handled like an unset one:
% tesseract --list-langs|head -3
List of available languages in "/opt/homebrew/share/tessdata/" (682):
Fraktur
Greek
% TESSDATA_PREFIX= tesseract --list-langs|head -3
List of available languages in "/opt/homebrew/share/tessdata/" (682):
Fraktur
Greek
@jonashaag, addAvailableLanguages starts searching for Tesseract model files at a well defined start location and is called recursively only if it finds a directory. Unless you have a cyclic directory structure, there won't be an infinite recursion. macOS and Linux use the same code.
Can you see the arguments of the stat64 calls? Where does the search start? Maybe there are simply a lot of files and directories below your start location. If there is really an endless recursion, it will have a first directory entry which will occur again and again.
Aha, interesting, the Homebrew version works fine. The conda-forge version that I've been using doesn't.
Output from macOS's fs_usage:
11:50:07.100355 open F=3 (R_____N____X) xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/.pixi/envs/default/share/tessdata 0.000020 tesseract.1362835
11:50:07.100364 fstatfs64 F=3 0.000002 tesseract.1362835
11:50:07.100439 getdirentries64 F=3 B=0x14d0 0.000074 tesseract.1362835
11:50:07.100443 stat64 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/.pixi/envs/default/share/tessdata 0.000003 tesseract.1362835
11:50:07.100450 open F=4 (R_____N____X) xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/.pixi/envs/default/share/tessdata 0.000007 tesseract.1362835
11:50:07.100451 fstatfs64 F=4 0.000001 tesseract.1362835
11:50:07.100501 getdirentries64 F=4 B=0x14d0 0.000050 tesseract.1362835
11:50:07.100503 stat64 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/.pixi/envs/default/share/tessdata 0.000002 tesseract.1362835
11:50:07.100509 open F=5 (R_____N____X) xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/.pixi/envs/default/share/tessdata 0.000006 tesseract.1362835
11:50:07.100510 fstatfs64 F=5 0.000001 tesseract.1362835
11:50:07.100559 getdirentries64 F=5 B=0x14d0 0.000049 tesseract.1362835
11:50:07.100561 stat64 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx/.pixi/envs/default/share/tessdata 0.000002 tesseract.1362835
[... repeated indefinitely ...]
There are no symlinks in .pixi/envs/default/share/tessdata or any of its children.
$ file --no-dereference .pixi/envs/default/share/tessdata/** | grep sym
[empty]
Conda links:
I am still searching for a description of the build process which was used for the Conda Tesseract package.
Thanks. I could build it locally, and the result works fine. It looks like the error is in libtesseract.5.dylib which is not surprising because that library contains the function addAvailableLanguages.
In my tests with the conda-forge binary I get some more strange results:
tesseract --list-langsworks if thetessdatadirectory is empty.- It also terminates if there is a single entry (file or directy) below
tessdata, but fails to report if that file is a model file. - It runs endless if there are two or more entries below
tessdata.
I cannot explain that strange results with the Tesseract code, and debugging without debug symbols is rather time consuming. Therefore I suggest to report the issue to conda-forge.
I think it is related to Conda's prefix replacement. The prefix ends up being empty or something like that. I will build a package with debug symbols/prints and check what's going on.
That's a good hint which helped me find the root of the problem.
Patching TESSDATA_PREFIX in the compiled library does currently not work for Tesseract because that string is assigned to std::string datadir in CCUtil::main_setup. This datadir gets a different string value after the patch, but its length is not patched and still has the value of the unpatched string. Therefore datadir won't get the required terminating '/' character.
This is not restricted to installations on macOS. Installations with Conda on Linux will have the same problem.
I think there is a simple fix which I will try later.
Pull request #4239 should fix this issue.
Fantastic! Thanks a million!
@jonashaag, I am sorry, but my patch does not work. I'll try a different fix tomorrow.
Pull request #4240 has a different fix which I now tested successfully on Linux and on macOS.