Can't get camera api to run

[Paul Sonnentag]

I'm not sure what I'm doing wrong, but the new camera api is not working for me.

This is my paper program:

// Cam
importScripts('paper.js');

(async () => {
  var canvas = await paper.get('canvas');
  var ctx = canvas.getContext('2d');

  setInterval(async () => {
    const papers = await paper.get('papers'); 
    const camera = await paper.get('camera');

    paper.drawFromCamera(ctx, camera, papers[2068].points, papers[757].points);
    ctx.commit();
  });
})();

Initially I got the error that the data is not transferable:

I'm wondering why this line doesn't have an await even though ImageCapture.grabFrame returns a Promise. Is that a mistake or am I missing something?

When I add a await I get a different Error from ImageCapture.grabFrame:

@shaunlebron Can you give me some pointers what I'm doing wrong. Thanks, I'm really excited to build something with the new camer API.

[shaun]

Thanks, there were some changes made before merge that I probably didn't check? Will look tonight

[shaun]

Good catch on the grabFrame issue. Fixed that here: #56

As for the last error about the Track being in an invalid state, I couldn't reproduce. Maybe the camera access is blocked? Here's where the projector page retrieves the video track of the camera:

paperprograms/client/projector/ProjectorMain.js

Lines 15 to 22 in a35abb9

 navigator.mediaDevices 
   .getUserMedia({ 
     audio: false, 
     video: cameraVideoConstraints, 
   }) 
   .then(stream => { 
     this._videoCapture = new ImageCapture(stream.getVideoTracks()[0]); 
   }); 

[Paul Sonnentag]

I still haven't fully resolved this issue. I could find out what's behind the "invalid state" error. According to the spec this error occurs if the readyState of the track is not "live". I couldn't figure out why this occurred though. Once I started logging the readyState of the track the problem no longer occurred. It seems that the implementation of Chrome is still buggy. Right now it works for me if I reload the projector from time to time.

Another issue that I've noticed is that the call to this.props.grabCameraImageAndProjectionData() would sometimes reject with undefined.

This error didn't affect the program immediately because I call get('camera') continuously with setInterval but long term this will probably lead to a memory leak:

  setInterval(async () => {
    const papers = await paper.get('papers');
    const camera = await paper.get('camera'); // if the call on the projector side fails this will block forever

     ....
   }, 100)

Has someone else experienced these problems?