infoforcefeed / OlegDB

Enough works to use this in production

Home Page:http://olegdb.org

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Transaction Optimizations

qpfiffer opened this issue · comments

Idle thoughts on ways to improve the transaction implementation:

  • dirty state: if a transaction didn't actually do anything (quite common for ol_unjar operations) then we should just discard the transaction when we're done instead of writing back an empty AOL file. Done.
  • Don't read the transaction's AOL file from disk: This is probably the slowest part of the whole procedure. I did it in a really dumb way. The transaction is written to disk, and then read back in from disk during commit.

Keeping all them in memory at all time isn't the best either, can't we have either a timeout or a "when I'm holding more than X transactions I'll put N oldest away" thing?

@Hamcha Wow I guess I never saw your comment, but yeah with the current architecture we should be able to do that.