Table.openBlock fails on Windows 7 but works on Debian

Question

Table.openBlock fails on Windows 7 but works on Debian

metroal opened this issue 6 years ago · comments

The following code (using release 0.11):

   public ChainstateIterator() {
        Options options = new Options();
        options.createIfMissing(false);
        options.compressionType(CompressionType.NONE);
        String datadir = treasure.getDataDir();

        try {
            db = factory.open(new File(datadir + "chainstate" + File.separator), options);
            obfuscationKey = extractKey(db);
            System.out.println(new Bytes(obfuscationKey));
            levelDBIterator = db.iterator();
            levelDBIterator.seekToFirst();
        } catch (IOException e) {
            ...
        }
    }

works perfectly on Debian Stretch, amd64 but fails with:

Exception in thread "main" java.lang.RuntimeException: java.nio.channels.ClosedChannelException
	at com.google.common.base.Throwables.propagate(Throwables.java:241)
	at org.iq80.leveldb.table.Table.openBlock(Table.java:83)
	at org.iq80.leveldb.util.TableIterator.getNextBlock(TableIterator.java:102)
	at org.iq80.leveldb.util.TableIterator.seekInternal(TableIterator.java:57)
	at org.iq80.leveldb.util.TableIterator.seekInternal(TableIterator.java:26)
	at org.iq80.leveldb.util.AbstractSeekingIterator.seek(AbstractSeekingIterator.java:41)
	at org.iq80.leveldb.util.InternalTableIterator.seekInternal(InternalTableIterator.java:45)
	at org.iq80.leveldb.util.InternalTableIterator.seekInternal(InternalTableIterator.java:25)
	at org.iq80.leveldb.util.AbstractSeekingIterator.seek(AbstractSeekingIterator.java:41)
	at org.iq80.leveldb.impl.Level.get(Level.java:134)
	at org.iq80.leveldb.impl.Version.get(Version.java:172)
	at org.iq80.leveldb.impl.VersionSet.get(VersionSet.java:223)
	at org.iq80.leveldb.impl.DbImpl.get(DbImpl.java:616)
	at org.iq80.leveldb.impl.DbImpl.get(DbImpl.java:577)
	at nxt.crosschain.db.ChainstateIterator.extractKey(ChainstateIterator.java:110)
	at nxt.crosschain.db.ChainstateIterator.<init>(ChainstateIterator.java:44)
	at nxt.crosschain.db.LevelDBImportWindowsTest.main(LevelDBImportWindowsTest.java:67)
Caused by: java.nio.channels.ClosedChannelException
	at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110)
	at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:721)
	at org.iq80.leveldb.table.FileChannelTable.read(FileChannelTable.java:96)
	at org.iq80.leveldb.table.FileChannelTable.readBlock(FileChannelTable.java:55)
	at org.iq80.leveldb.table.Table.openBlock(Table.java:80)
	... 15 more

on Windows 7, x86 using JDK 1.8.0_161
what could be the problem with nio.channels?

ivanicsjozsef94 · Answer 1 · Mon Mar 12 2018 04:58:37 GMT+0800 (China Standard Time)

Hi,
I'm getting similar exceptions on my Raspberry Pi. (My code works fine on Ubuntu and Windows.)
Caused by: java.lang.RuntimeException: java.nio.channels.ClosedChannelException at com.google.common.base.Throwables.propagate(Throwables.java:240) at org.iq80.leveldb.table.Table.openBlock(Table.java:83) at org.iq80.leveldb.util.TableIterator.getNextBlock(TableIterator.java:1 02) at org.iq80.leveldb.util.TableIterator.getNextElement(TableIterator.java :79) at org.iq80.leveldb.util.AbstractSeekingIterator.hasNext(AbstractSeeking Iterator.java:48) at org.iq80.leveldb.util.InternalTableIterator.getNextElement(InternalTa bleIterator.java:51) at org.iq80.leveldb.util.AbstractSeekingIterator.hasNext(AbstractSeeking Iterator.java:48) at org.iq80.leveldb.util.DbIterator.resetPriorityQueue(DbIterator.java:1 55) at org.iq80.leveldb.util.DbIterator.<init>(DbIterator.java:75) at org.iq80.leveldb.impl.DbImpl.internalIterator(DbImpl.java:778) at org.iq80.leveldb.impl.DbImpl.iterator(DbImpl.java:744) at org.iq80.leveldb.impl.DbImpl.iterator(DbImpl.java:735) at org.iq80.leveldb.impl.DbImpl.iterator(DbImpl.java:84) at com.hunelco.forint.database.LevelDBPeerStore.getAll(LevelDBPeerStore. java:127) ... 2 more Caused by: java.nio.channels.ClosedChannelException at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110) at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:720) at org.iq80.leveldb.table.FileChannelTable.read(FileChannelTable.java:96 ) at org.iq80.leveldb.table.FileChannelTable.readBlock(FileChannelTable.ja va:55) at org.iq80.leveldb.table.Table.openBlock(Table.java:80) ... 14 more

byant3125 · Answer 2 · Mon Dec 10 2018 15:45:48 GMT+0800 (China Standard Time)

The following code (using release 0.11):

   public ChainstateIterator() {
        Options options = new Options();
        options.createIfMissing(false);
        options.compressionType(CompressionType.NONE);
        String datadir = treasure.getDataDir();

        try {
            db = factory.open(new File(datadir + "chainstate" + File.separator), options);
            obfuscationKey = extractKey(db);
            System.out.println(new Bytes(obfuscationKey));
            levelDBIterator = db.iterator();
            levelDBIterator.seekToFirst();
        } catch (IOException e) {
            ...
        }
    }

works perfectly on Debian Stretch, amd64 but fails with:

Exception in thread "main" java.lang.RuntimeException: java.nio.channels.ClosedChannelException
	at com.google.common.base.Throwables.propagate(Throwables.java:241)
	at org.iq80.leveldb.table.Table.openBlock(Table.java:83)
	at org.iq80.leveldb.util.TableIterator.getNextBlock(TableIterator.java:102)
	at org.iq80.leveldb.util.TableIterator.seekInternal(TableIterator.java:57)
	at org.iq80.leveldb.util.TableIterator.seekInternal(TableIterator.java:26)
	at org.iq80.leveldb.util.AbstractSeekingIterator.seek(AbstractSeekingIterator.java:41)
	at org.iq80.leveldb.util.InternalTableIterator.seekInternal(InternalTableIterator.java:45)
	at org.iq80.leveldb.util.InternalTableIterator.seekInternal(InternalTableIterator.java:25)
	at org.iq80.leveldb.util.AbstractSeekingIterator.seek(AbstractSeekingIterator.java:41)
	at org.iq80.leveldb.impl.Level.get(Level.java:134)
	at org.iq80.leveldb.impl.Version.get(Version.java:172)
	at org.iq80.leveldb.impl.VersionSet.get(VersionSet.java:223)
	at org.iq80.leveldb.impl.DbImpl.get(DbImpl.java:616)
	at org.iq80.leveldb.impl.DbImpl.get(DbImpl.java:577)
	at nxt.crosschain.db.ChainstateIterator.extractKey(ChainstateIterator.java:110)
	at nxt.crosschain.db.ChainstateIterator.<init>(ChainstateIterator.java:44)
	at nxt.crosschain.db.LevelDBImportWindowsTest.main(LevelDBImportWindowsTest.java:67)
Caused by: java.nio.channels.ClosedChannelException
	at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110)
	at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:721)
	at org.iq80.leveldb.table.FileChannelTable.read(FileChannelTable.java:96)
	at org.iq80.leveldb.table.FileChannelTable.readBlock(FileChannelTable.java:55)
	at org.iq80.leveldb.table.Table.openBlock(Table.java:80)
	... 15 more

on Windows 7, x86 using JDK 1.8.0_161
what could be the problem with nio.channels?

same problem

Honore Vasconcelos · Answer 3 · Mon Dec 10 2018 16:24:03 GMT+0800 (China Standard Time)

In your log do you have a similar exception but with exception ClosedByInterruptException?

byant3125 · Answer 4 · Mon Dec 10 2018 16:26:59 GMT+0800 (China Standard Time)

In your log do you have a similar exception but with exception ClosedByInterruptException?

Exception in thread "main" java.lang.RuntimeException: java.nio.channels.ClosedChannelException
at com.google.common.base.Throwables.propagate(Throwables.java:240)
at org.iq80.leveldb.table.Table.openBlock(Table.java:83)
at org.iq80.leveldb.util.TableIterator.getNextBlock(TableIterator.java:102)
at org.iq80.leveldb.util.TableIterator.seekInternal(TableIterator.java:57)
at org.iq80.leveldb.util.TableIterator.seekInternal(TableIterator.java:26)
at org.iq80.leveldb.util.AbstractSeekingIterator.seek(AbstractSeekingIterator.java:41)
at org.iq80.leveldb.util.InternalTableIterator.seekInternal(InternalTableIterator.java:45)
at org.iq80.leveldb.util.InternalTableIterator.seekInternal(InternalTableIterator.java:25)
at org.iq80.leveldb.util.AbstractSeekingIterator.seek(AbstractSeekingIterator.java:41)
at org.iq80.leveldb.impl.Level.get(Level.java:134)
at org.iq80.leveldb.impl.Version.get(Version.java:172)
at org.iq80.leveldb.impl.VersionSet.get(VersionSet.java:223)
at org.iq80.leveldb.impl.DbImpl.get(DbImpl.java:616)
at org.iq80.leveldb.impl.DbImpl.get(DbImpl.java:577)
at org.iq80.leveldb.impl.DownToFile.downloadFromDb(DownToFile.java:33)
at org.iq80.leveldb.impl.DownToFile.main(DownToFile.java:21)
Caused by: java.nio.channels.ClosedChannelException
at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110)
at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:721)
at org.iq80.leveldb.table.FileChannelTable.read(FileChannelTable.java:96)
at org.iq80.leveldb.table.FileChannelTable.readBlock(FileChannelTable.java:55)
at org.iq80.leveldb.table.Table.openBlock(Table.java:80)
... 14 more
this is my log

zhenyu.yin · Answer 5 · Wed Dec 19 2018 10:37:12 GMT+0800 (China Standard Time)

@byant3125
Leveldb pure java version, no one has maintained, the problem is quite a lot, especially in the windows environment, please try the leveljni version, the problem is relatively less, the api interface is the same.

Dain Sundstrom · Answer 6 · Thu Jan 03 2019 09:55:41 GMT+0800 (China Standard Time)

I don't think LevelDB will work on Windows. The design requires atomic overwrite and I don't think you get that guarantee on windows file systems (at least from Java)

Mathias Herberts · Answer 7 · Fri Jun 14 2019 17:31:32 GMT+0800 (China Standard Time)

The issue seems to be related to the change that occurred back in 2015 in commit 3d9bc83 of TableCache.java

As the FileInputStream is being closed when the try block is exited, the FileChannelTable is left with a closed channel.

The auto close was introduced to avoid having a Too Many Open files error but it has side effects.

What I can't explain at the moment is that this does not cause any problem on some platforms.

Mathias Herberts · Answer 8 · Fri Jun 14 2019 17:55:38 GMT+0800 (China Standard Time)

From reading the TableCache code, the Too Many Open files errors seem to be mitigable by setting the maxOpenFiles option when creating a DB instance.

Mathias Herberts · Answer 9 · Fri Jun 14 2019 19:29:58 GMT+0800 (China Standard Time)

The reason why the code works in some systems and not others is due to the fact that on 64 bits systems the implementation of Table used is based on mmap and therefore does not attempt to access the closed FileChannel.