Segmentation fault on bulk insert
supersteves opened this issue · comments
Attempting to bulk insert using a simple script to do JSON based row by row insert from a CSV file. I'm trying to insert 1 billion rows, however every approx. 130k rows, the (master) node I'm connected to (of 4) has a seg fault and I have to manually restart that node (and the bulk insert).
What diagnostics can I retrieve for you?
I assume this is due to a known bug that was fixed on master some time ago (in 11da8ee) :/ However, we have not yet pushed the binary/stable release for 0.5.0, so you currently either need to build the source from master or download a recent snapshot build here: http://build.fnrd.net/artifacts/public/eventql-snapshot/release-20170227184144/
Also, we have just now added an "end-to-end" test that checks 10m inserts using the final compiled binaries to ensure that this works here https://github.com/eventql/eventql/blob/master/test/system/partitions/partitions.cc#L123 (the test is using the evqlctl table-import utility, which is also new in 0.5.0)
If using a 0.5.0 build doesn't solve the problem for you, could you run make test and see if any of the tests fail? Many thanks!
Paul, thanks - I'll get back to this soon and try 0.5. I'll close and re-open if it's still occurring.