ArrowBufPointer returns null buffer in the indexing thread

[Finn Völkel]

This happens while syncing an Auctiomark node. So there are no concurrent queries out. Querying only happens on the indexing thread.

15:01:33.283 [xtdb-tx-subscription-pool-1-thread-5] ERROR xtdb.indexer - error in indexer
java.lang.NullPointerException: Cannot invoke "org.apache.arrow.memory.ArrowBuf.getByte(long)" because the return value of "org.apache.arrow.memory.util.ArrowBufPointer.getBuf()" is null
	at xtdb.trie.HashTrie$Companion.bucketFor(HashTrie.kt:59)
	at xtdb.trie.HashTrie$Companion.compareToPath(HashTrie.kt:66)
	at xtdb.trie.EventRowPointer.<init>(EventRowPointer.kt:24)
	at xtdb.operator.scan.TrieCursor$fn__33875.invoke(scan.clj:255)
	at xtdb.operator.scan.TrieCursor.tryAdvance(scan.clj:243)
	at xtdb.operator.project.ProjectCursor.tryAdvance(project.clj:112)
	at xtdb.operator.rename.RenameCursor.tryAdvance(rename.clj:31)
	at xtdb.operator.project.ProjectCursor.tryAdvance(project.clj:112)
	at xtdb.query$wrap_cursor$reify__37702.tryAdvance(query.clj:71)
	at java.base/java.util.Spliterator.forEachRemaining(Spliterator.java:332)
	at xtdb.indexer$query_indexer$eval_query__37894.invoke(indexer.clj:371)
	at xtdb.indexer$wrap_xtql_args$fn__37929.invoke(indexer.clj:468)
	at xtdb.indexer$foreach_arg_row.invokeStatic(indexer.clj:388)
	at xtdb.indexer$foreach_arg_row.invokePrim(indexer.clj)
	at xtdb.indexer$$reify__37946.indexOp(indexer.clj:376)
	at xtdb.indexer.Indexer$fn__37969$index_tx_ops__37972$fn__37985.invoke(indexer.clj:607)
	at clojure.lang.AFn.call(AFn.java:18)
	at io.micrometer.core.instrument.AbstractTimer.recordCallable(AbstractTimer.java:175)
	at xtdb.indexer.Indexer$fn__37969$index_tx_ops__37972.invoke(indexer.clj:605)
	at xtdb.indexer.Indexer$fn__37969$fn__37993.invoke(indexer.clj:619)
	at xtdb.indexer.Indexer$fn__37969.invoke(indexer.clj:618)
	at xtdb.indexer.Indexer.indexTx(indexer.clj:580)
	at xtdb.log.watcher$watch_log_BANG_$reify__41611.acceptRecord(watcher.clj:53)
	at xtdb.log$tx_handler$fn__33260.invoke(log.clj:38)
	at clojure.core.protocols$iter_reduce.invokeStatic(protocols.clj:49)
	at clojure.core.protocols$fn__8246.invokeStatic(protocols.clj:75)
	at clojure.core.protocols$fn__8246.invoke(protocols.clj:75)
	at clojure.core.protocols$fn__8194$G__8189__8207.invoke(protocols.clj:13)
	at clojure.core$reduce.invokeStatic(core.clj:6895)
	at clojure.core$reduce.invoke(core.clj:6877)
	at xtdb.log$handle_polling_subscription$fn__33266.invoke(log.clj:52)
	at clojure.lang.AFn.run(AFn.java:22)
	at java.base/java.lang.Thread.run(Thread.java:1583)

[Finn Völkel]

The problem is that with compaction we might need to access multiple pages of the same VSR on one thread. With standard VSR caching this is not possible, as the vectors will change underneath our readers and that creates problems.