Upgrade from neo4j 2.2 to 3.0.7 fails

[Jenny Pettersson]

I am trying to upgrade my Neo4j 2.2.3 to 3.0.7. Importing the data seems to work but when I try to start the database server with dbms.allow_format_migration=true it failes with:

2016-11-23 17:10:40.461+0000 INFO  [o.n.k.i.s.p.StoreMigrator] Migrating Legacy indexes (2/3):
2016-11-23 17:10:40.777+0000 ERROR [o.n.k.i.s.p.LegacyIndexMigrator] Migration of legacy indexes failed. Index: geom__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__ can't be migrated. Migration of legacy index at path:C:\neo4j-3.7\data\databases\graph.db\upgrade\index\lucene\node\geom__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__ failed.
org.neo4j.upgrade.lucene.LegacyIndexMigrationException: Migration of legacy index at path:C:\neo4j-3.7\data\databases\graph.db\upgrade\index\lucene\node\geom__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__ failed.
	at org.neo4j.upgrade.lucene.LuceneLegacyIndexUpgrader.upgradeIndexes(LuceneLegacyIndexUpgrader.java:114)
	at org.neo4j.kernel.impl.storemigration.participant.LegacyIndexMigrator.migrateLegacyIndexes(LegacyIndexMigrator.java:121)
	at org.neo4j.kernel.impl.storemigration.participant.LegacyIndexMigrator.migrate(LegacyIndexMigrator.java:77)
	at org.neo4j.kernel.impl.storemigration.StoreUpgrader.migrateToIsolatedDirectory(StoreUpgrader.java:241)
	at org.neo4j.kernel.impl.storemigration.StoreUpgrader.migrateIfNeeded(StoreUpgrader.java:133)
	at org.neo4j.kernel.impl.storemigration.DatabaseMigrator.migrate(DatabaseMigrator.java:98)
	at org.neo4j.kernel.NeoStoreDataSource.upgradeStore(NeoStoreDataSource.java:565)
	at org.neo4j.kernel.NeoStoreDataSource.start(NeoStoreDataSource.java:434)
	at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:434)
	at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
	at org.neo4j.kernel.impl.transaction.state.DataSourceManager.start(DataSourceManager.java:99)
	at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:434)
	at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
	at org.neo4j.kernel.impl.factory.GraphDatabaseFacadeFactory.newFacade(GraphDatabaseFacadeFactory.java:140)
	at org.neo4j.kernel.impl.enterprise.EnterpriseFacadeFactory.newFacade(EnterpriseFacadeFactory.java:42)
	at org.neo4j.graphdb.EnterpriseGraphDatabase.<init>(EnterpriseGraphDatabase.java:57)
	at org.neo4j.server.enterprise.EnterpriseNeoServer.lambda$static$1(EnterpriseNeoServer.java:85)
	at org.neo4j.server.database.LifecycleManagingDatabase.start(LifecycleManagingDatabase.java:89)
	at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:434)
	at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:107)
	at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:187)
	at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:90)
	at org.neo4j.server.BlockingBootstrapper.start(BlockingBootstrapper.java:43)
	at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:67)
	at org.neo4j.server.enterprise.EnterpriseEntryPoint.start(EnterpriseEntryPoint.java:42)
Caused by: org.apache.lucene.index.IndexFormatTooOldException: Format version is not supported (resource BufferedChecksumIndexInput(MMapIndexInput(path="C:\neo4j-3.7\data\databases\graph.db\upgrade\index\lucene\node\geom__neo4j-spatial__LayerNodeIndex__internal__spatialNodeLookup__\segments_1"))): -11 (needs to be between 1071082519 and 1071082519). This version of Lucene only supports indexes created with release 4.0 and later.
	at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:299)
	at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:493)
	at org.apache.lucene.index.SegmentInfos$1.doBody(SegmentInfos.java:490)
	at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:731)
	at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:683)
	at org.apache.lucene.index.SegmentInfos.readLatestCommit(SegmentInfos.java:490)
	at org.apache.lucene.index.DirectoryReader.listCommits(DirectoryReader.java:257)
	at org.apache.lucene.index.IndexUpgrader.upgrade(IndexUpgrader.java:158)
	at org.apache.lucene.index.IndexUpgrader.main(IndexUpgrader.java:78)
	at org.neo4j.upgrade.lucene.IndexUpgraderWrapper.upgradeIndex(IndexUpgraderWrapper.java:66)
	at org.neo4j.upgrade.lucene.LuceneLegacyIndexUpgrader.upgradeIndexes(LuceneLegacyIndexUpgrader.java:109)
	... 24 more
2016-11-23 17:10:40.783+0000 INFO  [o.n.k.i.DiagnosticsManager] --- STOPPING diagnostics START ---
2016-11-23 17:10:40.783+0000 INFO  [o.n.k.i.DiagnosticsManager] --- STOPPING diagnostics END ---
2016-11-23 17:10:40.788+0000 INFO  [o.n.k.i.e.EnterpriseFacadeFactory] Shutdown started

I decided that I want to get rid of the spatial layer before I upgrade since I will be fine with using distance and point directly in cypher in Neo4j 3. The question is, how to get rid of it and is it even possible? I cannot see any suitable endpoints looking at the REST-api 😨 Is it safe to delete it manually via all RTREE_REFERENCE relationships?

[ehx-v1]

Looks like there's a too wide Lucene version jump between Neo4j 2.2 and 3.0.7 to migrate the database easily anyway. If I didn't misread, you'll probably have to delete all indexes, then migrate, and set up the indexes from scratch. And the indexes are not the Spatial RTree ones, but some different ones. You might want to think over whether that's really worth it.