Feature Request : NVR Tiered Storage

[briangunderson]

It would be nice to be able to have Scrypted NVR be able to use multiple tiers of storage locations. Blue Iris does this fairly well. Using tiered storage, NVR would save new clips to fast storage such as SSD, and when clips need to age out to make room for newer clips, rather than being deleted they are instead moved to secondary/tertiary disks which most often will be spinning disks. Indexes, snapshots, etc are ideally kept in primary storage along with the newest clips, this obviously helps with responsiveness. Thanks for your consideration!

[Koushik Dutta]

I was thinking about tiering storage by stream type. the low res stream needs to be on fast storage for timeline scrub. the high res stream can be on spinning disks. currently it cleans out high res streams before low res streams to retain space. but this is the opposite of what you want, as it is optimized for timeline responsiveness. need to think about this some more.

[briangunderson]

My thinking is that I would want super responsive high performance scrubbing to review events that took place recently (last hour? day?) and for events that took place a long time ago (weeks?) I would be more tolerant of a slow disk. That's just my opinion and what I got used to with blue iris. It might not be the best architecture but it makes sense to me.

…

On Mon, Mar 27, 2023 at 1:20 PM Koushik Dutta ***@***.***> wrote: I was thinking about tiering storage by stream type. the low res stream needs to be on fast storage for timeline scrub. the high res stream can be on spinning disks. currently it cleans out high res streams before low res streams to retain space. but this is the opposite of what you want, as it is optimized for timeline responsiveness. need to think about this some more. — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOO7SRLVJKUQ4XMRMECPSTW6HK7ZANCNFSM6AAAAAAWJKNABA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[Koushik Dutta]

scrypted scrubbing uses the low resolution stream. the contributors for scrub performance is the availability of said substream, and fast seek times on disk.

are you experiencing issues scrubbing right now?

[briangunderson]

Scrypted is light years ahead of anything else I've tried. It is FAST. That said, when I migrated from Blue Iris I didn't swap out my 3-tier storage solution for something more suitable for Scrypted's storage architecture, so I'm "stuck" using just my big slow 3.5" drives and forgoing the NVMe SSD and 15k SAS drives that previously served as my "fastest" and "fast" tiers. I'm a network engineer by trade and not a storage expert so it's quite possible that the architecture I'm envisioning isn't actually the architecture that would yield the best results. You seem to have a pretty good handle on tuning real-time digital video systems! :)

…

On Mon, Mar 27, 2023 at 2:45 PM Koushik Dutta ***@***.***> wrote: scrypted scrubbing uses the low resolution stream. the contributors for scrub performance is the availability of said substream, and fast seek times on disk. are you experiencing issues scrubbing right now? — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAOO7SRUC4AFKJ6ASBPIOXTW6HU5TANCNFSM6AAAAAAWJKNABA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[Koushik Dutta]

Gotcha, so this is more a feature request/question on how to utilize multiple slow vs fast disks. And maybe whether that’s even necessary. There’s a few sharding options worth investigating, ie retention vs responsiveness.

[Koushik Dutta]

This is now available in beta.
https://discord.com/channels/882329362295316500/933126942943739956/1149859852047368192