Doesn't handle non-ascii input characters?

Question

Doesn't handle non-ascii input characters?

jdbevan opened this issue 8 years ago · comments

Firstly, this is awesome 👍 :)

I cloned and compiled it fine on my MacBook Pro running OS X 10.10.5.

$ gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.2 (clang-700.1.81)
Target: x86_64-apple-darwin14.5.0
Thread model: posix

Then I ran tree | bin/nms and it looked great, then I pressed "any key" and once it had finished "decoding" I got the following output:

I'm using zsh:

$ zsh --version
zsh 5.2 (x86_64-apple-darwin14.5.0)

And this version of tree:

$ tree --version
tree v1.7.0 (c) 1996 - 2014 by Steve Baker, Thomas Moore, Francesc Rocher, Florian Sesser, Kyosuke Tokoro

And this is my LANG env variable:

$ echo $LANG
en_GB.UTF-8

It just dawned on me that the output of tree might not be ASCII compliant? I know you're restricting the nms output to ASCII only, but it should revert those characters back to the original input right?

Brian Barto · Answer 1 · Thu May 12 2016 05:00:13 GMT+0800 (China Standard Time)

I agree that supporting non-ascii characters on the input side would be ideal. I will be looking for a solution to this soon.

Brian Barto · Answer 2 · Fri May 13 2016 03:45:48 GMT+0800 (China Standard Time)

Okay, I built support for UTF-8 input. @jdbevan Can you please run:

cd no-more-secrets
git pull origin master
make
sudo make install

... and let me know if it is fixed on your end?

Thanks,
Brian

Jon Bevan · Answer 3 · Fri May 13 2016 16:45:27 GMT+0800 (China Standard Time)

Thanks @bartobri - I've hit a few problems...

I'm running on OS X and I don't have the ncursesw.h file - I've got ncurses installed with wide character support and having had a peek through the files that are installed I'm fairly confident that the ncurses.h file on my machine points to the ncursesw.h file.

Secondly, I'm running on x86_64 architecture, so when I changed the Makefile to use not look for ncursesw.h I got the following output:

gcc -Wextra -Wall -o bin/nms obj/nms.o obj/main.o
Undefined symbols for architecture x86_64:
"_beep", referenced from:
_nms_exec in nms.o
"_cbreak", referenced from:
_nms_exec in nms.o
"_curs_set", referenced from:
_nms_exec in nms.o
"_endwin", referenced from:
_nms_exec in nms.o
"_flushinp", referenced from:
_nms_exec in nms.o
"_getmaxx", referenced from:
_nms_exec in nms.o
"_getmaxy", referenced from:
_nms_exec in nms.o
"_has_colors", referenced from:
_nms_exec in nms.o
"_init_pair", referenced from:
_nms_exec in nms.o
"_initscr", referenced from:
_nms_exec in nms.o
"_noecho", referenced from:
_nms_exec in nms.o
"_scrollok", referenced from:
_nms_exec in nms.o
"_start_color", referenced from:
_nms_exec in nms.o
"_stdscr", referenced from:
_nms_exec in nms.o
"_waddnstr", referenced from:
_nms_exec in nms.o
"_wattr_off", referenced from:
_nms_exec in nms.o
"_wattr_on", referenced from:
_nms_exec in nms.o
"_wgetch", referenced from:
_nms_exec in nms.o
"_wmove", referenced from:
_nms_exec in nms.o
"_wrefresh", referenced from:
_nms_exec in nms.o
"_wscrl", referenced from:
_nms_exec in nms.o
ld: symbol(s) not found for architecture x86_64

Brian Barto · Answer 4 · Sat May 14 2016 00:55:37 GMT+0800 (China Standard Time)

This looks like there is something wrong with your ncurses installation. These errors are are similar to what I would expect to see if you were missing the ncurses.h header file and library (not ncursesw), which my program still uses. Along with the ncurses installation, you also need to have ncurses wide character support (ncursesw) installed which is either included with the original ncurses package (as it is with Fedora, Redhat, and Arch linux) in which case you don't need to do anything beyond installing ncurses, or you have to install it as a separate package (as it is with Ubuntu) in which case you install ncursesw along side of ncurses so both exist. I guess it depends on how the package maintainers decide to package it up as to what steps you need to take. What facility are you using to install ncurses? Have you installed the developer tools from apple? I thought ncurses was included with that but I may be incorrect.

Also make sure you run "make clean" before you run "make" just to be sure you remove any possibly issues with old object files.

Brian Barto · Answer 5 · Tue May 17 2016 01:18:05 GMT+0800 (China Standard Time)

Did some googling around there there seems to be a lot of people saying that if you install xcode along with the command line utilities (part of xcode?) then you should have the ncurses library.

You can also try installing ncurses via homebrew by doing the following:

brew install ncurses
brew doctor
brew link ncurses # might require a --force

Just throwing out possibilities. If you happen to figure out the right recipe please tell me so I can update my documentation. Unfortunately I don't have a mac to figure it out myself.

Jon Bevan · Answer 6 · Tue May 17 2016 20:30:47 GMT+0800 (China Standard Time)

I managed to solve it! I hadn't specified the correct lib directory for ncurses. I had installed ncurses via: brew install ncurses

I just needed to change the Makefile to look like this (for my homebrew configuration):

LDLIBS = -lncursesw -L/opt/boxen/homebrew/opt/ncurses/lib/
NCURSES_H = /opt/boxen/homebrew/opt/ncurses/include/ncurses.h

Thanks for the fix!

Brian Barto · Answer 7 · Thu May 19 2016 22:12:47 GMT+0800 (China Standard Time)

Okay great. Glad it is all working and that there is nothing wrong in the code. I am going to assume that the package maintainer for brew has to routinely override these directories when making packages, and will do so for no-more-secrets after I tag another release that includes UTF-8 support.

Closing issue.