Background-Compaction fails
mapbased opened this issue · comments
I got following exception when datasize about to 110G, abount 1800,000,000 rows.
need help,thanks
Options options = new Options();
options.createIfMissing(false);
options.compressionType(CompressionType.SNAPPY);
DBFactory factory = Iq80DBFactory.factory;
DB db;
try {
db = factory.open(from, options);
} catch (IOException e) {
throw new java.lang.RuntimeException(e);
}
org.iq80.leveldb.impl.DbImpl$BackgroundProcessingException: java.lang.NullPointerException
at org.iq80.leveldb.impl.DbImpl.checkBackgroundException(DbImpl.java:411)
at org.iq80.leveldb.impl.DbImpl.get(DbImpl.java:572)
at org.iq80.leveldb.impl.DbImpl.get(DbImpl.java:565)
............................................................................................
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.NullPointerException
at org.iq80.leveldb.impl.Compaction.totalFileSize(Compaction.java:129)
at org.iq80.leveldb.impl.Compaction.isTrivialMove(Compaction.java:120)
at org.iq80.leveldb.impl.DbImpl.backgroundCompaction(DbImpl.java:468)
at org.iq80.leveldb.impl.DbImpl.backgroundCall(DbImpl.java:426)
at org.iq80.leveldb.impl.DbImpl.access$100(DbImpl.java:83)
at org.iq80.leveldb.impl.DbImpl$2.call(DbImpl.java:396)
at org.iq80.leveldb.impl.DbImpl$2.call(DbImpl.java:390)
fixed myself by change max levels from 7 to 10
Change max levels from 7 to 10: I've encountered the same error. Do you have a link to a page describing how to configure levels in leveldb?
I have similar error. What are other consequences of changing max levels? How about performance?
It seems changing max levels was just a workaround. This PR potentially fixes the root cause: #80