Invalid byte sequence

Question

Invalid byte sequence

skatenerd opened this issue 6 years ago · comments

skatenerd commented 6 years ago

When I try to download a file that contains an "ñ" character, I get an invalid byte sequence error:

view $ inproc "curl" ["https://www2.census.gov/geo/relfiles/cdsld13/35/co_cd_35.txt"] empty

Is there a way to encourage this to somehow start using unicode?

Gabriella Gonzalez · Answer 1 · Fri Aug 17 2018 22:57:36 GMT+0800 (China Standard Time)

@skatenerd: Yes. You can use the following command to specify that your program should assume a UTF-8 encoding globally:

GHC.IO.Encoding.setLocaleEncoding System.IO.utf8

skatenerd · Answer 2 · Sat Aug 18 2018 06:21:31 GMT+0800 (China Standard Time)

Thanks! Turns out the file in question had a really strange encoding, and it wouldnt play nicely with any tooling (bash or turtle) until I ran it through iconv!

Gabriella Gonzalez · Answer 3 · Sat Aug 18 2018 08:53:32 GMT+0800 (China Standard Time)

@skatenerd: You're welcome! 🙂