Reader doesn't work with streams
MrGussio opened this issue · comments
I've been trying to implement your library myself for a livestream instead of a single image, but I'm unable to get it to work with the qrcode as reader. EAN for example works fine, but when I try to add the qrcode to the readers settings, it will crash upon seeing a code (any code, not specific QR codes), trying to call the function decodePattern(), which apparently doesn't exist yet for the QR code reader object.
Source code of the App.tsx: https://gist.github.com/MrGussio/4631583720c9ab00dfe8264966a23ab3
That's interesting, for sure, thanks for trying that out -- I hadn't quite got to that point yet, as I hit a stumbling block with live stream and workers.. but i hadn't tried it again now that i've just plain disabled workers across the entire library, since they didn't really work well with the existing feature set, let alone new features.
Probably worth also trying the existing tests with just the qrcode reader, to see if that might be the source of the problem as well -- i think the tests that i have written so far, enable both code128 and qrcode, to test that a built-in reader doesn't get broken by adding an external reader. That could also be checked by trying the LiveStream with the code128 reader also enabled when qrcode is enabled.
It would only moderately surprise me if the detection code is going down different paths when it's in live stream mode versus non.
If you happen to get a chance to try out either of those cases, before I do, let me know the results ..
... could you also point to a sample of the qrcode you're trying to scan? i'm trying to trace back where it might be on a different path, to see if the place where i plugged in the qrcode reader is correct or not, and i'm finding that if BarcodeLocator returns a successful result when it calls getBoundingBoxes() then it should ignore the qrcode locator.. which seems to make sense.. except there might be some false positive in BarcodeLocator, which could maybe make weird things happen.
I'm not sure exactly what the correct solution is at a quick glance -- maybe QrCodeReader should extend BarcodeReader .. maybe it should implement a stub decodePattern.. maybe tryDecode() should check that the reader has a decodePattern function before calling it...
Thanks for your quick reply!
I wasn't even able to test it, because upon loading the reader it already gave this error. When I tried doing this with an already existing reader in Quagga (I tried it with EAN and some barcodes on groceries, and code128 as well) and it wouldn't give this error immmediately, but only after a code is actually presented to the camera. I think the QrCodeReader somehow thinks it detected a code in my face or something, and then tried to decode it, but failed because it couldn't find the function.
I already tried going through the source code of the library and trying to find out what this function does, and as you already mentioned it is included in BarcodeReader, but I didn't get much further then that.
The only thing I found weird is that the original setup, with just decoding a static image, does work for both examples (as your test cases succeed as well), but the live example isn't working. Maybe there's something wrong in the initialization of the livestream decoder? Because I assume you're using the same function for decoding an image from a stream as decoding a static image.
Okay, after giving it a few more tries, it gives me some different behaviour:
The QrCodeReader fully crashes after a code is displayed to it, and gives the following error:
I can't do much with it because it refers to the quagga.min.js file, which is hard to debug due to the minimalistic file size.
Testing it with an EAN code again doesn't actually crash the reader, and it does render some green boxes on the canvas that it detects a code, but it never actually shows me a result (and it never renders the blue box with the red line).
Really weird that it does this just 10 minutes later without really changing anything big in the configuration.
My current source file of App.tsx: https://gist.github.com/MrGussio/7c2772511bee52969f7f9e02d9c47b58
assume you're using the same function for decoding an image from a stream as decoding a static image.
See, I thought that would be the case too, but since decodeSingle() works, and the live stream handling doesn't, it seems that decodeSingle() doesn't go through BarcodeDecoder.tryDecode()..
On the really bright side, I just found a USB webcam that actually is capable of getting an image good enough for Quagga to decode, so I hopefully won't have to ship code to a server and access it from my cell phone to be able to make meaningful changes to code that affects live stream. Unfortunately, I am also really busy trying to secure some new employment this year. :|
decodeSingle goes init() then start() - start() goes to update() which goes to locateAndDecode(), locateAndDecode goes to getBoundingBoxes(), if bounding boxes are found, it goes to decodeFromBoundingBoxes(), if no bounding box is found, then it goes to decodeFromImage()
decodeFromBoundingBoxes() does a decodeFromBoundingBox() on each box given, which does a tryDecode() and if that fails, a tryDecodeBruteForce().
SO, it looks to me, like your qr code is triggering getBoundingBoxes, so Quagga somehow thinks that there's a potential barcode area, which is something that the sample images in the tests here don't seem to do. Otherwise, I'm not seeing a split between the code paths.
The divergence between decodeSingle() and a LiveStream, seems to be that Live calls startContinuousUpdate() instead of update(), and startContinuousUpdate() kicks off a repeating call to update(). There's a bunch of stuff in update() to handle workers, but now that i've just straight disabled workers, it should end up in the exact same place both in LiveStream and in decodeSingle() -- inside locateAndDecode().
I think that, perhaps we should try adding a stub decodePattern to the QR reader, and see if that changes the behavior for the better or not. I'm not sure exactly which the best solution is, but that should at least get it moving past the first hurdle. I'll do a commit on that here in a moment. Maybe this evening I'll get a chance to plugin this new camera to my dev machine and give the livestream a spin, if not, it might not be til next weekend, unless i find some time inbetween
In any case, even if this suddenly makes it work right, the output from the QR readers is not going to be nearly as robust as the barcode reader output - I haven't gone to implementing anything more than the bare minimum possible result output, this is still just a proof to validate that the ability to do something with the image, if all configured barcode readers fail first, works.
Alright, I get your point!
I also tried implementing a stub method somewhere to see what it would do, but I didn't really manage to implement it correctly, kept giving me some weird TypeScript error or something.
Which tools are you currently using to develop these projects?
I'm not really a JS developer, I just use it in the frontend every now and then, but have never really used it for such big projects, maybe I could take a look into it coming weekend as well.
The way I'm currently able to debug the Quagga library in my own implementation is using the webcam on my laptop, but I also got it running on my phone by letting my PHP library host to the 0.0.0.0 address, which makes my phone able to connect to it as well using the local IPv4 of my computer on the network. The only thing I had to do is add that local IP to the unsafely-treat-insecure-origin-as-secure flag in Chrome, else it wouldn't ask permission for my camera. Maybe this will help you to debug as well, it worked quite well for me.
I use VSCode for editing, and all the dev tools are listed in package.json dependencies and devDependencies. I've so far avoided intentionally touching anything that would affect the LiveStream areas, because there's no good automated test methods for that section of code at the moment.
I either implement a test directly in the main library, or into this library, depending on how easily i can fit it into the structure of the main libs automated tests, or if it's easier to do with the more modern tests here.. then i npm run build, and npm run test on the main, or npm run e2e here.
I'm definitely still working on updating a lot of things, the tools in the main repo were so out of date when I started, that I've spent quite a lot of time trying to just update dependencies, because the javascript ecosystem has become so much easier to work with over the last couple of years.
I probably really should implement some kind of test using the ImageStream function, which should hopefully work as a pretty reasonable replacement for being able to test the LiveStream .. testing LiveStream seems like a pretty difficult proposition, so ImageStream could mock it.
Basically, the process is:
write new code
write test for new code
build library
run tests
Once I'm happy with how that's working, I publish an update, then update the version used in my real-world application that uses LiveStream, and make sure it doesn't break.
oh, yeah, pull the latest version of this lib into your app, and see if that changes the error situation?
Upon scanning a QR code now, it returns me the following error:
It's the same error as I've already commented above (where the "t was undefined")
For other codes (I tried this with just EAN but I assume it's for all the other existing ones) it does still seem to work, it will render the green detection boxes, but it won't actually call the onDetect method.
well, it's forward progress. i'll see what I can come up with soon, I haven't really had a chance to dig in with this new camera yet.. at least i'll probably get to figure out how to run and debug the non-minified version, that might be helpful.. haven't had to do that before
Hi, is work qr reader with livestream? I try it on react example, only register new reader and add it to array. I try it on phone but qrcode don't scan, no error in console
my code https://gist.github.com/sneznaovca/9162cba070346fe6ab838556136872f0
This is still quite experimental, my guess is no, but i've been concentrating my rare development time on the typescript conversion and modernization of the core library, because I'm pretty sure there's changes that need to be made somewhere to get this to work all the way through.
Curious that you're getting no errors, and MrGussio was getting an error when it read.
What I'd do with yours, @sneznaovca is put some logging in handleProcessed() and onDetected(), to see if either are ever getting triggered.
I literally haven't tried it on a livestream input yet, it was something I was hoping to get to sooner than this, but I became unemployed at the beginning of the year, so survival has become more important than code.
Per ericblade/quagga2#240 (comment)
I'm going to close this until someone says it's still an issue. Thanks for everyone's help!!!