Quick-start from README errors on Windows 10

[Jeroen Heijmans]

Saw the associated blog post linked on Hacker News, and was happy to see the source available. Thanks a bunch!

I tried the quick start from the README to see if I could get it to work locally, but failed to do so. For what it's worth, here's a way to reproduce my issue:

Environment:

• NPM 8.1.2
• Node v16.13.2
• Windows 10 21H1
• Chrome Version 98.0.4758.102 (Official Build) (64-bit)
• navigator.appVersion in the Electron app that starts: "5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.122 Electron/9.4.4 Safari/537.36"
• Run from Powershell, version 5.1.19041.1320

Steps taken:

• git clone the repository
• npm i
• npm start
• Press "Click to Start"

Result:

• warnings and errors on the Electron dev tools console (see below)
• no relevant output in the main window
• an error on the console that booted the electron app

Detailed output from Powershell window:

C:\git\oss\unredacter [main ≡]> npm run start

> unredacter@1.0.0 start
> npm run build && electron ./dist/main.js

> unredacter@1.0.0 build
> tsc

(node:15124) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(node:14468) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
Error occurred in handler for 'redact': RangeError [ERR_OUT_OF_RANGE]: The value of "offset" is out of range. It must be >= 0 and <= 5244. Received -960
    at boundsError (internal/buffer.js:72:9)
    at Buffer.readUInt32BE (internal/buffer.js:295:5)
    at Jimp.<anonymous> (C:\git\oss\unredacter\node_modules\@jimp\plugin-crop\dist\index.js:43:37)
    at scan (C:\git\oss\unredacter\node_modules\@jimp\utils\dist\index.js:53:9)
    at Jimp.scanQuiet (C:\git\oss\unredacter\node_modules\@jimp\core\dist\index.js:1262:32)
    at Jimp.cropQuiet (C:\git\oss\unredacter\node_modules\@jimp\plugin-crop\dist\index.js:42:12)
    at Jimp.<computed> [as crop] (C:\git\oss\unredacter\node_modules\@jimp\core\dist\index.js:1176:23)
    at C:\git\oss\unredacter\dist\main.js:363:47
    at step (C:\git\oss\unredacter\dist\main.js:52:23)
    at Object.next (C:\git\oss\unredacter\dist\main.js:33:53) {
  code: 'ERR_OUT_OF_RANGE',
  methodName: 'crop',
  eventName: 'error'
}

Output on the Electron dev tools (last part appears after pressing the start button):

DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/preload.js.map: 
4DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/index.js.map: 
DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/phash.js.map: 
DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/request.js.map: 
DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/index.js.map: 
DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/constants.js.map: 
DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/composite-modes.js.map: 
DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/promisify.js.map: 
DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/mime.js.map: 
DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/image-bitmap.js.map: 
6DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/index.js.map: 
DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/iq.js.map: 
3DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/index.js.map: 
DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/blur-tables.js.map: 
14DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/index.js.map: 
DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/measure-text.js.map: 
DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/index.js.map: 
DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/resize.js.map: 
DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/resize2.js.map: 
4DevTools failed to load SourceMap: Could not load content for file:///C:/git/oss/unredacter/index.js.map: 
undefined:1 GET file:///C:/git/oss/unredacter/undefined net::ERR_FILE_NOT_FOUND
Image (async)
(anonymous) @ renderer.ts:54
step @ renderer.ts:6
(anonymous) @ renderer.ts:6
(anonymous) @ renderer.ts:6
__awaiter @ renderer.ts:6
gatherResults @ renderer.ts:36
(anonymous) @ renderer.ts:24
postMessage (async)
(anonymous) @ renderer.ts:60
VM55 renderer_init.js:75 Uncaught (in promise) Error: Error invoking remote method 'redact': RangeError [ERR_OUT_OF_RANGE]: The value of "offset" is out of range. It must be >= 0 and <= 5244. Received -960
    at EventEmitter.o.send.o.invoke (VM55 renderer_init.js:75)

Screenshot of the main window:

Of course my ultimate goal is to replace secret.png with my own image and try it out, but for now I'm stuck at the default setup.

---

PS. I'd absolutely understand if you'd close this issue as "wontfix", given that you provide the source code as a courtesy and companion to a blog post. But just in case you'd be glad to get some feedback on others trying your solution, I figured I'd take a moment to write this issue.

Thanks again for a wonderful blog post and for sharing the code! 👍

[Dan Petro]

Oh, you know what I've seen this before on Mac. What's happening is a strange OS compatibility issue. For some reason Chrome renders the text totally wrong on non-Linux. I'm not totally sure why.

The reason it breaks on that crop is that it renders the text to a page and then tries to crop it down. But the image is a totally different size than expected and so it dies.

One way to debug is to stick a file write in the code...

image.writeAsync(path.join(__dirname, "../test_plaintext.png"));

At main.ts at around line 210. That will still crash, but it will spit out an image to file right before it does. Try posting that image

[Huhni]

This is what the result looks like for me on Windows 11:

[Dan Petro]

So there's at least two issues, one is that the window is too narrow vertically. It's set right here https://github.com/BishopFox/unredacter/blob/main/src/main.ts#L171

You can try increasing that but... 100px ought to be enough.

Secondly, the font is just wrong. You can see what it looks like on my machine.

Is Sans-Serif maybe not a default Windows font? Chrome will backup to some other font if you type gibberish there, so maybe that's what's happening

[Huhni]

A height of 120 fixes the OUT_OF_RANGE error for me and keeps looping through from a to z.
I don't know what to do about the font though, afaik Chrome uses Arial as default for sans-serif.

This seems like a promising project though. I'm sure one could create some gpu accelerated version of this trying to solve an image with a number of common fonts and sizes at the same time. Maybe even heuristics on common character combinations or even a neural network could be used to improve it even more.

[Dan Petro]

Chrome on Linux apparently uses a different sans-serif font than Arial. It's just showing up as "custom"

Which is not helpful. I'm not really sure what font this is now.

[Dan Petro]

I just changed the default font to Arial, works on my machine. :) See if it works on yours now

[Huhni]

It still just iterates from a to z, maybe because I have to set the height to 120? Also without a dark theme enabled the window looks like the screenshot from the first post, white text on white background.

[Dan Petro]

oh blah, right. 692144e should have a fix for that

[Dan Petro]

oh, though actually it's supposed to iterate from a-z for a while. Maybe this isn't obvious from the interface. The interface is... lacking. But the first thing it does is auto-discover the x and y offsets. So you'll see it go through all the characters and eventually find offset [2, 5] as "good". And then once it goes through all the offsets, it will actually do the solution guessing.

So maybe just try waiting a bit longer. If it blows past [2, 5] without adding it to the "Good offsets to try" list, then there's something wrong

[Huhni]

Oh I see. After a while it added [3,5] to the list, resulting in the text vu. Maybe differences in font-rendering are the reason it doesn't work right, stuff like ClearType/DirectWrite, grayscale vs subpixel aliasing and so on... Also the reason why css fingerprinting works so well.

I was able to solve my own redacted text, so that's probably it. The performance could really be better though, but it's a nice proof-of-concept.

[Dan Petro]

There must be some os-compatiblity thing going on. If you add that line of code I mentioned above, then maybe we can see better what that is:

image.writeAsync(path.join(__dirname, "../test_plaintext.png"));

As for performance, absolutely. I'm quite sure that this whole thing could be significantly faster. For one, Electron might not be the best environment. The whole thing is CPU-bound and single-threaded. About half way through I wished that I had done it in Go, so I could use some nice and speedy goroutines for the enumeration. But c'est la vie.

[Huhni]

This is the final result for me:

[Webinx]

Same issue to me, Windows 10.
Height 120 doesn't fix the OUT_OF_RANGE error in my case.

In which OS it was tested to work?

[MING]

Same issue on mac 🙈

plus

DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.

[Dan Petro]

Same issue to me, Windows 10. Height 120 doesn't fix the OUT_OF_RANGE error in my case.

In which OS it was tested to work?

Vanilla Ubuntu 20.04 works. It's looking like cross-compatibility is quite hard. Unredacter needs the window and font and everything to line up exactly the same, so minor inconsistencies compound.

[Jeroen Heijmans]

Yeah, that's totally fair. As I said in the original post: it might be just fine to either close the issue, or let it sit until someone from the community feels like adding cross-OS compatibility. Whatever you'd choose, I'd totally understand, and think it's already awesome that you open sourced the code companion to your blogpost! ❤️

[Talinn92]

@dan-bishopfox

Hey I don't mean to be off-topic or anything but for some reason I can't tag you in that thread. You worked on another project, untwister, which you released a long time ago but there seems to be a major issue with the CPU usage on it? If you have time, could you help me figure this out? altf4/untwister#47