character encoding

Question

character encoding

TinManTex opened this issue 7 years ago · comments

Running vol1_main0_def_v00.parts through foxtool and searching the resulting fox2.xml for
Chargin
shows

        <property name="partName" type="String" container="StaticArray" arraySize="1">
          <value>Charginï½‡Up</value>
        </property>

The equivalent in vol0_main0_def_v00.parts provides

        <property name="partName" type="String" container="StaticArray" arraySize="1">
          <value>ChargingUp</value>
        </property>

vol parts files.zip

My current theory (on assumptions) is a tpp developer accidentally added a Japanese character, that KJPs fox2 compiler is likely to be inputting shift jis encoded files, but what we are seeing with FoxTool is due to it's use of default encoding?

Atvaark · Answer 1 · Sun Sep 17 2017 19:39:43 GMT+0800 (China Standard Time)

The G appears to be a UTF-8 encoded fullwidth character. The developer probably pressed the wrong button on their keyboard.

Check if Release v0.2.5 fixes this issue and doesn't break anything else.

TinManTex · Answer 2 · Mon Sep 18 2017 08:42:49 GMT+0800 (China Standard Time)

Seems good playing with other fox files run through the updated foxtool. Infinite Heaven is up to a couple of hundred modified or new fox files, but it would have only been a mix of a dozen or so loaded at the points I was testing.
Haven't tested with recompiled since it's just for once specific section of one mission that's annoying to get to. The only reason I noticed it was because I was string scraping and running through hashing function which understandably choked on the invalid characters.

Also noticed you updated your CityHash package but haven't committed to your repo?

Thanks again for your tools, my mod wouldn't have gotten as far without them.