UTF-16BE not exported correctly

[suckerp]

HI,

I'm trying to export a JSON to later import into SQL Server. For that I need to use a UTF-16BE encoded file.

My problem is that when I try to convert it to UTF-16BE (utf16be) I get ANSI encoded file with only NUL entries. When I convert it into utf16 (le) it works correctly.
Here's what I'm doing.

//to write the data from the arry into a UTF-8 File I use the standard writeFile function.
fs.writeFile(filename, JSON.stringify(data, null, 4), 'utf8')

//Input opens the previously written UTF-8 encoded file and the it should be converted into UTF-16BE but here's where I run into problems
fs.createReadStream(input)
.pipe(iconv.decodeStream('utf8'))
.pipe(iconv.encodeStream('utf16be'))
.pipe(fs.createWriteStream(output))

Like I said before when I use something like utf16 or utf16le instead of utf16be I get the correct output file for that encoding. But not for utf16be.

Is anybody else having the same issue with the UTF-16BE encoding?

[Alexander Shtuchkin]

Can it be that the viewer you use does not support utf16-be? I see there are letters between NUL-s in your screenshot which is what I'd expected if that was a correct utf16-be file that was opened as utf8. Separately, to help programs recognize this as a utf16-be, you can try adding BOM - there's a parameter to do that in encodeStream. Alex

…

On Mon, Jul 3, 2023, 08:42 suckerp ***@***.***> wrote: HI, I'm trying to export a JSON to later import into SQL Server. For that I need to use a UTF-16BE encoded file. My problem is that when I try to convert it to UTF-16BE (utf16be) I get ANSI encoded file with only NUL entries. When I convert it into utf16 (le) it works correctly. Here's what I'm doing. //to write the data from the arry into a UTF-8 File I use the standard writeFile function. fs.writeFile(filename, JSON.stringify(data, null, 4), 'utf8') //Input opens the previously written UTF-8 encoded file and the it should be converted into UTF-16BE but here's where I run into problems fs.createReadStream(input) .pipe(iconv.decodeStream('utf8')) .pipe(iconv.encodeStream('utf16be')) .pipe(fs.createWriteStream(output)) [image: grafik] <https://user-images.githubusercontent.com/15850780/250558648-a888e20b-d28b-483d-8973-2dde427efc1e.png> Like I said before when I use something like utf16 or utf16le instead of utf16be I get the correct output file for that encoding. But not for utf16be. Is anybody else having the same issue with the UTF-16BE encoding? — Reply to this email directly, view it on GitHub <#309>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEZKHP3IDU5VXROBLNB5QLXOK443ANCNFSM6AAAAAAZ4M6EEY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>