Disk-backed structures that can be larger than available RAM
These classes are designed to emulate a regular array and have the following features:
- they are a fixed length
- data can be accessed in a random manner
- the arrays are Read/Write
- all extend the ITitanicArray<T> interface
A basic disk-backed array using just a file and and LRU based cache.
This array is backed by a MemoryMapped file. Pretty much just a wrapper around the MemoryMappedFile class
This array is backed by a SQLite database.
This array is backed by a LiteDB database.
This array is backed by a RaptorDb database.
This array is backed by Magick.NET
Note: TitanicIMArray sometimes has a problem converting data to float values (the internal storage type). So this array might be unreliable.
Testing on my machine using 2^20 doubles (4MiB of data) results in these times:
- TitanicMMFArray
- Random Sequence: 3s
- Linear Sequence: 2s
- TitanicIMArray
- Random Sequence: 10s
- Linear Sequence: 9s
- TitanicRaptorDBArray
- Random Sequence: 13s
- Linear Sequence: 12s
- TitanicFileArray
- Random Sequence: 25s
- Linear Sequence: 12s
- TitanicSQLiteArray
- Random Sequence: 345s
- Linear Sequence: 294s
- TitanicLiteDBArray
- Random Sequence: ?s (took too long)
- Linear Sequence: 570s