Cannot get snapshot from nest batter powered doorbell
zeldamaster911 opened this issue · comments
I see that support for the new battery powered doorbell is in home assistant now, and I was stoked until I realized I couldn't take a snapshot. Is there a way I'm just missing? Calling camera.snapshot doesn't work, even with the stream already live. Neither does a round about method using the camera's token through the API. Am I missing anything?
Hi, the camera does not have a live feed, so it can't take snapshots. The old cameras are the same way, however the hack we had was taking 1 frame from the live RTSP feed. We are not doing WebRTC server side for battery cams. (yet! it's a complex protocol)
See https://next.home-assistant.io/integrations/nest/#camera for a written version of what I just said, in the beta docs.
However! You can take snapshots in response to an event and it should work within 30 seconds of the event happening.
I have also proposed a service to take snapshots in response to an event: home-assistant/core#58369 so its more explicit than relying on the camera snapshot which is not guaranteed to work.
Lastly, I am working through how to support mp4 preview animated clips for battery camera doorbell events. Those are super cool and should give you better events than the old static images for wired cams. Stay tuned there.
One last thing: You can set a camera card mode to "live" I think and it will show you still images from the stream. But basically, that's constantly starting webrtc connections which will probably drain the battery quickly so I am not sure I would recommend it.
You say that I can take a snapshot in response to an event. As in, an automation with a trigger of "Doorbell button pressed", or using Home Assistant's event system and nest_events? Is there any real difference between those?
Also, I appreciate the response and the hard work you've put into getting nest functional in Home Assistant.
Yes, you can use a automation with a trigger of doorbell button pressed. The event is more low level, but it's actually what makes the trigger happen under the covers.
(The new API for event snapshots may only support events I now realize, because it needs a nest_event_id, which I don't think is a variable in the trigger, but i'll figure that out and post a snippet in the docs when its ready.)
Thank you for the nice words!
@zeldamaster911 I've added support for doorbell media snapshots via a new media player in the latest beta of home assistant, however it still needs some work. Of interest to you is there is a new api path /api/nest/event_media/{device_id}/{event_id} where media can be fetched.
Right now you will get the simple 10 frame mp4 file without any transcoding to slow it down, which i'll work on later. I am curious if you could try this out and see how it works. I plan to iterate on this and make the media player more useful (e.g. with a disk backed storage)
I've been trying to test this out, but I'm a bit confused. There is no {event_id} included in any of the info from a nest_event when I'm listening in the developer tools section, so what exactly is it referencing? Shouldn't it be {nest_event_id} ? I apologize in advance for my lack of knowledge. I tried using the new api path in an automation for sending the mp4 video as a notification, but I keep getting errors about that path not existing.
service: notify.mobile_app_test data: title: motion detected message: front door motion detected data: video: /api/nest/event_media/{device_id}/{nest_event_id}
Yes, if you take the device_id and nest_event_id you can use those. I think if you want to actually use it in an automation you have to do something more complex to pull it from the trigger. Check out https://www.home-assistant.io/docs/automation/templating/#event
or this example here https://community.home-assistant.io/t/understanding-event-data-content/180171/5?u=allenporter
So maybe when its time to fetch the media the actual path is /api/nest/event_media/... device id .../{{ trigger.eventdata.nest_event_id }}
I don't know the syntax for getting the device id dynamically but maybe that can be hard coded from the automation temporarily until we figure that out.
Also, this only works in dev or beta release.
I've done some moderate testing of the new API path, and it works great for stills from my older model nest cam (RTSP), but when applying the same tests to try and get that 10 frame mp4 file, I always get a 404 error in response. I've also noticed that the nest event id's look very different when they come from events started by the doorbell
From my doorbell: "1631420517"
From my Nest Cam: "AVPHwEs__kz25CIIKql9MT9HeJHlkJwTppu5Yevc2UvFMzdM255dyNjz1PAVHJHiKSanaxRfKsxcoqhvIdBH24PE5iEfBw"
I noticed that trying to view event media in the media browser in home assistant, there was quite a delay from the time there was an entry for a given event, and the point at which the media would actually be viewable; with this in mind I tried adding in delay before trying to use the API path to grab the mp4 clip, but it would still end with a 404 response.
Am I still missing something?
Yes the events are different. My impression is that is not a problem.
Are you saying you got the /api/nest/event_media/ path working?
My impression is that we block publishing the event until after the media is fetched (for better or worse!).
If you have an automation you'd like to share, that would make that a little easier to discuss. I was playing with automations this weekend, but was mostly looking at images since android doesn't support rendering videos.
Also, FWIW, I am working on persisting these to disk automatically home-assistant/core#61641 since the current in memory behavior isn't great. Also have an opt out PR in flight.
Are you saying you got the
/api/nest/event_media/path working? My impression is that we block publishing the event until after the media is fetched (for better or worse!).
Yeah, I've been using it for checking out who's at the door based on motion and person detection:
`alias: Entryway Cam detects motion, notify pixel with image
description: ''
trigger:
- platform: event
event_type: nest_event - platform: device
device_id: dd27f97953dc5d127776b4866ccfcbd6
domain: nest
type: camera_motion
condition: []
action: - delay:
hours: 0
minutes: 0
seconds: 5
milliseconds: 0 - service: notify.living_room_tv
data:
message: Someone is at the door!
data:
image:
path: /api/camera_proxy/camera.entryway - service: notify.notify
data:
message: There was motion detected
title: Motion Detected
data:
image: >-
/api/nest/event_media/(MyNestCamDeviceID)/{{trigger.event.data.nest_event_id}}
video: >-
/api/nest/event_media/(MyDoorBellDeviceID)/{{trigger.event.data.nest_event_id}}
mode: single
`
This automation sends a notification to all phones in the household (my partner has an Iphone which supports videos in notifications) and she receives the notification, but the video isn't included. It actually has an image in its place that says "Media Not Found"
I've also been dabbling with Node-red and the API node that's included in the custom home assistant integration, and using that I can also retrieve images from my Nest Cam, but not videos from the doorbell.
If you're adding the ability to store all media on disk though, I imagine this would solve all my problems. Thanks, man!
I've added a table in https://www.home-assistant.io/integrations/nest/#camera that explains all the supported options for every camera. Notably, a camera can't support both image and video clips at the same time.
The home assistant beta was released today, and the documentation for upcoming features can be found here before it becomes released:
https://next.home-assistant.io/integrations/nest/#media-attachments
Generally, the fully featured media player now exists with APIs for attaching media to events and it has been documented with some use cases. You can get mp4 clips or a gif, etc.
Feel free to report issues against home assistant if you run into any, and it will get routed to me.