suminb / finance

경제적 자유로의 여정

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Batch insertion (with psycopyg2) is way too slow

suminb opened this issue · comments

We need to insert a large quantity of AssetValue records, but it's too slow. Takes more than 60sec to insert tens of rows.

This may be helpful...

Or we could use a thread pool to make things faster. We also could send all the insertion queries in a single batch (within a single transaction) but that will greatly increase the likelihood of failures, as we will try to insert data over overlapping periods most of the time.

Partially resolved by 315d0e8.