Substream toggle feature

Question

Substream toggle feature

rschitz opened this issue a year ago · comments

Hi and thanks for this great plugin.
I'd need to stream the substream instead of the main stream to save bandwidth and cpu, could you add this feature please?

Raphaël SCHITZ commented a year ago

thx a lot

vectronic · Answer 1 · Tue May 02 2023 14:48:03 GMT+0800 (China Standard Time)

Let me have a look when I find some free time in the next couple of weeks.

Greg · Answer 2 · Mon May 08 2023 22:06:57 GMT+0800 (China Standard Time)

love this plugin! would also like to request this feature, here is the relevant doc for the API call:

http://xxx.xxx.xxx.xxx/[API KEY]/toggleSubstream/[GROUP KEY]/[MONITOR ID]

https://hub.shinobi.video/articles/view/xm9HJFXI1XITt1y

vectronic · Answer 3 · Tue May 09 2023 23:19:57 GMT+0800 (China Standard Time)

I've had a look at that article above and re-read the request here - but to be honest, I have no idea what you're asking for nor do I understand how the substream functionality is supposed to work!

Are you saying you want a switch exposed in Homebridge to toggle the sub stream feed?

What is the different endpoint to read the stream from?

Currently I connect the streaming delegate either:

directly to the rtsp exposed by the camera (if auto_host URL is populated in monitor config embedded in the monitor details API response) i.e. this is the "Connection => Full URL Path" in the Monitor configuration screen if "Connection => Automatic" is true.
otherwise, the stream exposed by Shinobi in the the monitor details API response. I have only ever seen a single stream exposed in the streams array property here, so I always take the first one.

I configured sub-stream for one of my monitors and didn't see any difference in the API response except "subStreamActive" was set to true. And now that it is configured in the monitor settings, I can't remove it :-).

Raphaël SCHITZ · Answer 4 · Wed May 10 2023 00:14:10 GMT+0800 (China Standard Time)

Hi and thank you for your time on this topic.
the thing is i dont want the mainstream on my phone because of bandwidth limitation, only the low resolution substream but i still need the mainstream to be recorded on the server side.
right now to work around this i created another monitor only low res for homebridge sharing whitch is not ideal

Greg · Answer 5 · Wed May 10 2023 00:15:22 GMT+0800 (China Standard Time)

You need to run dashboard v3 and set it to the advanced view to see ALL the necessary sections for substream. The basic idea is that you can use the lower quality (ie less bandwidth and CPU) substream to view when using the dashboard (or a mobile device).

I could configure this by setting up a 2nd monitor that is only the substream and piping that to homekit, leaving the Main stream for recording only (never use to view) and not piping it to homekit, but then i need to setup a bunch of duplicate settings between the 2 monitors.

vectronic · Answer 6 · Wed May 10 2023 16:16:00 GMT+0800 (China Standard Time)

Ok. starting to understand. I need to have another go as I didn’t see any changes in the api response when I set up a substream. But maybe i didn’t configure it correctly.

I’ll have another look in the weekend.

vectronic · Answer 7 · Mon May 22 2023 01:59:20 GMT+0800 (China Standard Time)

I found some time to play around with this (I find the Shinobi docs, UI and API quite confusing).

I'm writing my plans below as much for my own benefit as anyone else's - but please let me know if you have any comments.

From what I see, if a monitor is configured with "Only When Watching, Use Substream" as per https://hub.shinobi.video/articles/view/xm9HJFXI1XITt1y then the stream listed in the monitor API response will be the substream and not the main stream. And it will only be live/available if the monitor API response also includes "subStreamActive": true.

If instead it returns "subStreamActive": false it is necessary to first do a GET request to http://xxx.xxx.xxx.xxx/[API KEY]/toggleSubstream/[GROUP KEY]/[MONITOR ID] which should return:

{
   "ok": true
}

and after this the status in the monitor API response will be "subStreamActive": true.

It looks like regardless of stream/substream/"only while watching" etc. there is always only a single stream URL provided in the monitor API response streams property. So I will continue to use this.

The only change I think I need to make is when the HomeKit streaming delegate prepareStream request occurs the accessory will need to make the GET request to:

http://xxx.xxx.xxx.xxx/[API KEY]/toggleSubstream/[GROUP KEY]/[MONITOR ID]

if the monitor API response returns "subStreamActive": false

I propose providing a new per monitor boolean configuration property use_dynamic_substream (defaulting to false) like this:

            "monitors": [
                {
                    "monitor_id": "rear",
                    "use_dynamic_substream": true
                },

If this is true the new logic for checking the subStreamActive state and performing the additional API request will be invoked whenever a new HomeKit streaming delegate prepareStream request occurs.

Note that there is already logic in the accessory stream delegate that will use the direct camera stream if possible if the monitor API response includes an auto_host value. Code is here https://github.com/vectronic/homebridge-shinobi/blob/master/src/shinobiStreamingDelegate.ts#L69-L80

In this case the HomeKit accessory will stream directly from the camera instead of from the Shinobi stream. If the accessory is in this "direct" mode, then the new logic will not be applied (as the stream comes directly from the camera, not Shinobi and thereby avoids the need for a Shinobi ffmpeg process entirely).

I'll scrape some more time in the next week to make the changes above.

vectronic · Answer 8 · Mon May 29 2023 00:44:51 GMT+0800 (China Standard Time)

This is implemented in version 3.2.0 if anyone is still interested. I have not tested it out as I am using direct camera streams as mentioned in the last comment.

If anyone wants to test it out, the best approach is to run Homebridge with debug logging e.g. homebridge -D and let me know of any errors occurring and if so also provide logs.

If I don't hear back in a couple of weeks I will close the ticket assuming it is a. unused or b. used and working flawlessly :-)

Raphaël SCHITZ · Answer 9 · Mon May 29 2023 00:48:52 GMT+0800 (China Standard Time)

thanks a lot for the effort, i'll test asap (was going to answer your last msg but postpone for when i got a little time to check the api and time flies you know the story)

Raphaël SCHITZ · Answer 10 · Tue Jun 06 2023 05:45:31 GMT+0800 (China Standard Time)

it's working but i get the primary link, not the Substream. i guess because the substream is "not active" as not beeing requested since it's on demand. On the grid, you have to request a substream switch before getting it and replacing the main feed:

So maybe you'd need to request the substream feed or to initiate the switch before?

vectronic · Answer 11 · Tue Jun 06 2023 14:50:36 GMT+0800 (China Standard Time)

Did you configure the monitor in the homebridge accessory config with use_dynamic_substream?

The logic is in place to request the toggle of the substream.

Can you send me debug logs?

Raphaël SCHITZ · Answer 12 · Wed Jun 07 2023 16:13:59 GMT+0800 (China Standard Time)

yep

logs coming

Raphaël SCHITZ · Answer 13 · Wed Jun 07 2023 16:41:42 GMT+0800 (China Standard Time)

[6/7/2023, 10:37:49 AM] [Shinobi] handleSnapshotRequest() success
[6/7/2023, 10:37:51 AM] [Shinobi] handleSnapshotRequest: Oi2rSRnZj7 => {"height":360,"width":640} from http://10.97.2.86:8080/xxxxxxxxxxxxxxxxxxx/jpeg/nIsKAgAhHq/Oi2rSRnZj7/s.jpg
[6/7/2023, 10:37:51 AM] [Shinobi] handleSnapshotRequest() success
[6/7/2023, 10:37:52 AM] [Shinobi] prepareStream: JLaikH6jxT => {"sessionID":"2ab9c628-43ec-4f2e-ba18-e02b0e0f7b25","targetAddress":"10.97.2.112","addressVersion":"ipv4","video":{"port":52814,"srtpCryptoSuite":0,"srtp_key":{"type":"Buffer","data":[173,226,70,210,80,4,135,164,200,154,192,186,250,151,125,156]},"srtp_salt":{"type":"Buffer","data":[25,175,106,46,37,5,210,189,197,142,69,88,241,72]}},"audio":{"port":57212,"srtpCryptoSuite":0,"srtp_key":{"type":"Buffer","data":[93,32,12,27,239,136,199,60,249,43,11,255,145,95,198,25]},"srtp_salt":{"type":"Buffer","data":[54,251,163,193,137,247,242,191,117,8,200,165,98,61]}}}
[6/7/2023, 10:37:52 AM] [Shinobi] fetching from Shinobi API: http://10.97.2.86:8080/xxxxxxxxxxxxxxxxxxx/monitor/nIsKAgAhHq/JLaikH6jxT
[6/7/2023, 10:37:52 AM] [Shinobi] fetching from Shinobi API: http://10.97.2.86:8080/xxxxxxxxxxxxxxxxxxx/toggleSubstream/nIsKAgAhHq/JLaikH6jxT
[6/7/2023, 10:37:52 AM] [Shinobi] substream toggle response: {"ok":true}
[6/7/2023, 10:37:52 AM] [Shinobi] prepareStream() success
[6/7/2023, 10:37:53 AM] [Shinobi] handleStreamRequest: {"sessionID":"2ab9c628-43ec-4f2e-ba18-e02b0e0f7b25","type":"start","video":{"codec":0,"profile":1,"level":2,"packetizationMode":0,"width":640,"height":360,"fps":30,"pt":99,"ssrc":728712149,"max_bit_rate":132,"rtcp_interval":0.5,"mtu":1378},"audio":{"codec":"OPUS","channel":1,"bit_rate":0,"sample_rate":24,"packet_time":60,"pt":110,"ssrc":2790607641,"max_bit_rate":24,"rtcp_interval":5,"comfort_pt":13,"comfortNoiseEnabled":false}}
[6/7/2023, 10:37:53 AM] [Shinobi] requested video stream: 640x360, 30 fps, 132 kbps, 1378 mtu
[6/7/2023, 10:37:53 AM] [Shinobi] -fflags +genpts -i rtsp://login:passwd@10.97.2.85:554/cam/realmonitor?channel=6&subtype=0 -vsync drop -vcodec copy -an -f rtp -payload_type 99 -ssrc 1891076 -srtp_out_suite AES_CM_128_HMAC_SHA1_80 -srtp_out_params reJG0lAEh6TImsC6+pd9nBmvai4lBdK9xY5FWPFI srtp://10.97.2.112:52814?rtcpport=52814&localrtcpport=52814&pkt_size=1378
[6/7/2023, 10:37:53 AM] [Shinobi] handleStreamRequest() START success
[6/7/2023, 10:37:53 AM] [Shinobi] ffmpeg received first frame

Raphaël SCHITZ · Answer 14 · Wed Jun 07 2023 16:43:01 GMT+0800 (China Standard Time)

rtsp://login:passwd@10.97.2.85:554/cam/realmonitor?channel=6&subtype=0 is the main stream not the substream:

vectronic · Answer 15 · Thu Jun 08 2023 19:27:39 GMT+0800 (China Standard Time)

Thanks for the info (luckily I just saw it in my spam folder...). I will have a look in the coming week (or 2) and update accordingly. When I tried myself it seemed I was getting the stream from the correct place - but maybe I was confused (or have different config).

vectronic · Answer 16 · Mon Jun 26 2023 17:16:27 GMT+0800 (China Standard Time)

@rschitz the latest version I pushed to NPM should be working better.

While fixing things I have realised a few things....

The current (new) behaviour is:

CONFIG

if dynamic sub-stream enabled:
   if direct sub-stream URL found in monitor config
      use direct sub-stream URL
   else
      use the first proxy provided by shinobi in monitor config
else
   if direct stream URL found in monitor config
      use direct stream URL
   else
      use the first proxy provided by shinobi in monitor config


CONNECT

if dynamic sub-stream enabled:
  check substream enable state and toggle to on if required   
connect to configured URL


DISCONNECT

if dynamic sub-stream enabled:
  check substream enable state and toggle to off if required

Realisations:

If the monitor config includes a direct camera sub-stream URL or camera stream URL then this is used. I think these will always exist and therefore the Shinobi proxy stream URL is NEVER used. I think this is preferable as it means the camera streams direct to the homebridge accessory where ffmpeg relays it (if possible with encode/decode). If the shinobi stream was used then the camera stream would go to shinobi (ffmpeg) then to homebridge (ffmpeg). However maybe there is something provided by the Shinobi stream which people want (i.e. overlays, masks etc.)? I'm not sure. If this is the case I guess I need to provide config to allow the forced used of the shinobi proxy stream.
Based on the last point, without using the Shinobi streams, the homebridge plugin doesn't actually use Shinobi much apart from using it as a source of camera URLs... I mean I guess it means cameras only need to be configured in Shinobi and not homebridge as well....
If the shinobi stream is never used (and instead the camera stream or sub-stream is always used) there is no need for the logic I introduced to toggle the sub-stream. Or at least it should only be used if use of the shinobi proxy stream is used.
When a stream disconnect occurs, I toggle off the substream process in Shinobi - I can't be aware if anything outside the Homebridge plugin is making use of this stream i.e. if someone is viewing the stream in the Shinobi UI at the same time. So in this case if you stop watching the stream in HomeKit, it will also stop streaming in Shinobi UI. I can make sure this is only an issue when using the shinobi proxy stream in the plugin (which will likely be never, unless I introduce a force option), but in this scenario I don't know how to determine if the proxy stream is in use by the Shinobi web UI.

Anyway, if you've read this far and still using the plugin or have any interest let me know your thoughts.

I think I will just aim to perform the toggle logic only when using the Shinobi proxy stream and then leave it like that for now....

Raphaël SCHITZ · Answer 17 · Mon Jun 26 2023 19:03:44 GMT+0800 (China Standard Time)

Hi @vectronic and thank you for the effort and the explanations. From what i understood from Shinobi doc ans UI, the alt stream is here if you have bandwith issue on the live grid.
I thought you could trigger that same request in the api but if not dont waste anymore time on this, i can live with extra cam in my setup it's fine.
Thanks a lot

vectronic · Answer 18 · Mon Jun 26 2023 22:24:27 GMT+0800 (China Standard Time)

@rschitz try the new version out - I think it should be doing what you want i.e. using rtsp://login:passwd@10.97.2.85:554/cam/realmonitor?channel=6&subtype=1

Raphaël SCHITZ · Answer 19 · Fri Jul 14 2023 20:27:11 GMT+0800 (China Standard Time)

works perfectly, thank you VERY much for your support!

vectronic · Answer 20 · Mon Aug 07 2023 03:31:36 GMT+0800 (China Standard Time)

Updated version to v4.0.0 to change as follows:

fix: modify support for substream to only use shinobi dynamic sub-stream support when using shinobi proxied streams
breaking: rename config use_dynamic_substream to use_substream

External functionality should not be different, but configuration needs to be changed from use_dynamic_substream to use_substream.