gchq / CyberChef

The Cyber Swiss Army Knife - a web app for encryption, encoding, compression and data analysis

Home Page:https://gchq.github.io/CyberChef

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Bug report: Output shows raw format while data is UTF-8 thus showing incorrect data in UI

marc-van-driel opened this issue · comments

Describe the bug
Output shows raw format while data is UTF-8 thus showing incorrect data in UI. Need to manually change Output Format Encoding to UTF-8 to see correct data. This happens with or without a recipe to process the data

Even trying to force we encode and decode text doesn't change this behavior.

To Reproduce
Steps to reproduce the behavior or a link to the recipe / input used to cause the bug:

Scenario 1:

  1. Leave everything on default (input and output on raw, no recipe)
  2. In Input open a UTF-8 file with some UTF-8 specific content (in my test case a Russian name)
  3. In Output look at content and content is shown incorrectly
  4. Change manually to UTF-8. Now shows correctly

Scenario 2:

  1. Leave everything on default (input and output on raw, no recipe)
  2. In Input open a UTF-8 file with some UTF-8 specific content (in my test case a Russian name)
  3. Process recipe that includes decode and encode text step to force UTF-8
  4. In Output look at content and content is shown incorrectly
  5. Change manually to UTF-8. Now shows correctly

Scenario 3:

  1. Leave everything on default (input and output on raw, no recipe)
  2. In Input open a UTF-8 file with some UTF-8 specific content (in my test case a Russian name) that was encoded (in this case base64)
  3. Process recipe that includes decodes the base64
  4. In Output look at content and content is shown incorrectly (even if we try to add a encode step_
  5. Change manually to UTF-8. Now shows correctly

Expected behavior
When file is processed using a recipe and the format is a specific encoding automatically adjust output character encoding to match correct format. If this is not possible have new special "flow" or "util" step to force specific encoding in UI.
Tried to force it by using decode text and encode text but that didn't work.
I also saved the resulting output and then opened that without any recipe and I also get inconsistent behavior. You can see the result in the below screenshots. The input show the data in UTF-8 although input format encoding shows Raw Bytes and the output shows the data as raw bytes and the output format encoding shows Raw Bytes

Screenshots
Show opened file after save of output

Show opened file after save of output - Update to utf8

Desktop (if relevant, please complete the following information):

  • OS: Windows
  • Browser: Edge version 121.0.2277.83 (also tested with Chrome)
  • CyberChef version: 10.8.0