Giters

orbitdb / orbitdb

Peer-to-Peer Databases for the Decentralized Web

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Error: Want for bafyreighfb3szinnqrktnuyriifexl2bcgpisbrlbaxfs4kgqp6ocwjbmy aborted

T00rk opened this issue · comments

commented

I was not able to open a database on two nodes using your example so I looked in issues and found #1154.
I've copied his code with little updates (fixes you proposed in the discussion).
I get exactly the same errors he got.

node:internal/process/promises:289
            triggerUncaughtException(err, true /* fromPromise */);
            ^

[AggregateError: All promises were rejected] {
  [errors]: [
    Error: Want for bafyreighfb3szinnqrktnuyriifexl2bcgpisbrlbaxfs4kgqp6ocwjbmy aborted
        at EventTarget.<anonymous> (file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/ipfs-bitswap/dist/src/notifications.js:56:24)
        at [nodejs.internal.kHybridDispatch] (node:internal/event_target:807:20)
        at EventTarget.dispatchEvent (node:internal/event_target:742:26)
        at abortSignal (node:internal/abort_controller:369:10)
        at AbortController.abort (node:internal/abort_controller:391:5)
        at EventTarget.onAbort (file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/any-signal/dist/src/index.js:8:20)
        at [nodejs.internal.kHybridDispatch] (node:internal/event_target:807:20)
        at EventTarget.dispatchEvent (node:internal/event_target:742:26)
        at abortSignal (node:internal/abort_controller:369:10)
        at AbortController.abort (node:internal/abort_controller:391:5),
    AggregateError: unable to fetch raw block for CID bafyreighfb3szinnqrktnuyriifexl2bcgpisbrlbaxfs4kgqp6ocwjbmy from any gateway
        at TrustlessGatewayBlockBroker.retrieve (file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/@helia/block-brokers/dist/src/trustless-gateway/broker.js:52:15)
        at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
        at async file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/@helia/block-brokers/dist/src/utils/networked-storage.js:182:31
        at async Promise.any (index 1)
        at async raceBlockRetrievers (file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/@helia/block-brokers/dist/src/utils/networked-storage.js:179:16)
        at async NetworkedStorage.get (file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/@helia/block-brokers/dist/src/utils/networked-storage.js:91:27)
        at async BlockStorage.get (file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/helia/dist/src/storage.js:68:20)
        at async Object.get (file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/@orbitdb/core/src/storage/ipfs-block.js:61:19)
        at async Object.get (file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/@orbitdb/core/src/storage/composed.js:52:15)
        at async Object.get (file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/@orbitdb/core/src/manifest-store.js:24:19) {
      [errors]: [
        Error: fetching raw block for CID bafyreighfb3szinnqrktnuyriifexl2bcgpisbrlbaxfs4kgqp6ocwjbmy from gateway https://dweb.link/ipfs/bafyreighfb3szinnqrktnuyriifexl2bcgpisbrlbaxfs4kgqp6ocwjbmy?format=raw was aborted
            at TrustlessGateway.getRawBlock (file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/@helia/block-brokers/dist/src/trustless-gateway/trustless-gateway.js:71:23)
            at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
            at async TrustlessGatewayBlockBroker.retrieve (file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/@helia/block-brokers/dist/src/trustless-gateway/broker.js:25:31)
            at async file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/@helia/block-brokers/dist/src/utils/networked-storage.js:182:31
            at async Promise.any (index 1)
            at async raceBlockRetrievers (file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/@helia/block-brokers/dist/src/utils/networked-storage.js:179:16)
            at async NetworkedStorage.get (file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/@helia/block-brokers/dist/src/utils/networked-storage.js:91:27)
            at async BlockStorage.get (file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/helia/dist/src/storage.js:68:20)
            at async Object.get (file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/@orbitdb/core/src/storage/ipfs-block.js:61:19)
            at async Object.get (file:///home/romain/Projets/orbitdb-nodejs-local-demo/node_modules/@orbitdb/core/src/storage/composed.js:52:15)
      ]
    }
  ]
}

Node.js v20.9.0

The code looks like this

import { gossipsub } from "@chainsafe/libp2p-gossipsub";
import { bootstrap } from "@libp2p/bootstrap";
import { mdns } from "@libp2p/mdns";
import { IPFSAccessController, createOrbitDB } from "@orbitdb/core";
import { LevelBlockstore } from "blockstore-level";
import { createHelia, libp2pDefaults } from "helia";
import { createLibp2p } from "libp2p";
import { join } from "path";

const main = async () => {
  if (process.argv.length < 4) process.exit(1);

  const [, , PORT, DIRECTORY, DATABASE = "FOOBAR"] = process.argv;

  const config = {
    ...libp2pDefaults(),
    addresses: {
      listen: [`/ip4/0.0.0.0/tcp/${PORT}`, "/webrtc"],
    },
    peerDiscovery: [
      mdns(),
      bootstrap({
        list: ["/ip4/127.0.0.1/tcp/8181", "/ip4/127.0.0.1/tcp/8282"],
      }),
    ],
    services: { pubsub: gossipsub({ allowPublishToZeroPeers: true }) },
  };

  console.log(`running with port:${PORT} and ${DIRECTORY}`);

  const ipfsDirectory = join("./", DIRECTORY, "helia");
  const orbitDirectory = join("./", DIRECTORY, "orbit");

  const blockstore = new LevelBlockstore(ipfsDirectory);

  const libp2p = await createLibp2p(config);
  const ipfs = await createHelia({ libp2p, blockstore });
  const orbitdb = await createOrbitDB({
    ipfs,
    id: DIRECTORY,
    directory: orbitDirectory,
  });

  const clientEventsDb = await orbitdb.open(DATABASE, {
    type: "events",
    syncAutomatically: true,
    AccessController: IPFSAccessController({ write: ["*"] }),
  });

  console.log("clientEventsDb address", clientEventsDb.address);
  console.log(await clientEventsDb.all());

  let index = 0;

  const sayHello = async () => {
    await clientEventsDb.add(`hello from ${DIRECTORY} ${++index}`);
    setTimeout(sayHello, 2000);
  };

  clientEventsDb.events.on("join", async (peerId, heads) => {
    console.log("peer", peerId);
  });

  clientEventsDb.events.on("update", async (entry) => {
    console.log("entry", entry.payload);
  });

  await sayHello();

  process.on("SIGINT", async () => {
    await clientEventsDb.close();
    await orbitdb.stop();
    await ipfs.stop();

    process.exit();
  });
};

main();

I first start the script with this command line

node server.js 8181 USER1

It creates a DB /orbitdb/zdpuAnwe1J3Y6oXHVxdvAMXVzn7tRrjKaHGmBdzDLfFXRMCEY then in another terminal

node server.js 8282 USER2 /orbitdb/zdpuAnwe1J3Y6oXHVxdvAMXVzn7tRrjKaHGmBdzDLfFXRMCEY
commented

Thanks. Just confirming #1163 ?

commented

I've tried using a LevelDataStore but that doesn't solve the issue. I don't know if both issues are related.

commented

@T00rk I will take a look at the script you have provided.

commented

I have run the script and launched two nodes. The problem appears to be that neither peer finds the other and connects. Since USER2 can't find USER1, OrbitDB can't find the db in order to replicate it.

libp2pDefaults() seems to be the cause of the issue. I suspect the problem is with the service settings (although I haven't tested enough to rule out other problems). I've also noticed that peer performance is degradated with default settings (I also notice the first peer eventually crashes with an unrelated libp2p error).

To solve the problem and get the two peers swapping db records, I added the following to the script:

  config.services = { 
    identify: identify(), 
    pubsub: gossipsub({ allowPublishToZeroPeers: true })
  }
  
  config.addresses.listen = [`/ip4/0.0.0.0/tcp/${PORT}`]

  config.peerDiscovery = [ 
    mdns(), 
    bootstrap({
      list: ["/ip4/127.0.0.1/tcp/8181", "/ip4/127.0.0.1/tcp/8282"],
    })]
commented

I've tried your solution. It doesn't work.

I've tried manual connection using libp2p dial method:

const { status } = await libp2p.dial(multiaddr("/ip4/127.0.0.1/tcp/8181"));
console.log("Connection", status);

It seems second node can connect to the first one as I get this output :

node server.js 8282 USER2 /orbitdb/zdpuAnwe1J3Y6oXHVxdvAMXVzn7tRrjKaHGmBdzDLfFXRMCEY
running with port:8282 and USER2
Connection open
commented

Could you please provide the full script with my modifications?

commented

EDIT:

I've added a LevelDataStore to Helia and it works.

import { gossipsub } from "@chainsafe/libp2p-gossipsub";
import { bootstrap } from "@libp2p/bootstrap";
import { identify } from "@libp2p/identify";
import { mdns } from "@libp2p/mdns";
import { multiaddr } from "@multiformats/multiaddr";
import { IPFSAccessController, createOrbitDB } from "@orbitdb/core";
import { LevelBlockstore } from "blockstore-level";
import { LevelDatastore } from "datastore-level";
import { createHelia, libp2pDefaults } from "helia";
import { createLibp2p } from "libp2p";
import { join } from "path";

const main = async () => {
  if (process.argv.length < 4) process.exit(1);

  const [, , PORT, DIRECTORY, DATABASE = "FOOBAR"] = process.argv;

  const config = {
    ...libp2pDefaults(),
    addresses: {
      listen: [`/ip4/0.0.0.0/tcp/${PORT}`],
    },
    peerDiscovery: [
      mdns(),
      bootstrap({
        list: ["/ip4/127.0.0.1/tcp/8181", "/ip4/127.0.0.1/tcp/8282"],
      }),
    ],
    services: {
      pubsub: gossipsub({ allowPublishToZeroPeers: true }),
      identify: identify(),
    },
  };

  console.log(`running with port:${PORT} and ${DIRECTORY}`);

  const ipfsBSDirectory = join("./", DIRECTORY, "helia", "BS");
  const ipfsDSDirectory = join("./", DIRECTORY, "helia", "DS");
  const orbitDirectory = join("./", DIRECTORY, "orbit");

  const blockstore = new LevelBlockstore(ipfsBSDirectory);
  const datastore = new LevelDatastore(ipfsDSDirectory);

  const libp2p = await createLibp2p(config);
  const ipfs = await createHelia({ libp2p, blockstore, datastore });

  const id = +DIRECTORY.replace("USER", "");
  if (id > 1) {
    const { status } = await libp2p.dial(multiaddr("/ip4/127.0.0.1/tcp/8181"));
    console.log("Connection", status);
  }

  const orbitdb = await createOrbitDB({
    ipfs,
    id: DIRECTORY,
    directory: orbitDirectory,
  });

  const clientEventsDb = await orbitdb.open(DATABASE, {
    type: "events",
    syncAutomatically: true,
    AccessController: IPFSAccessController({ write: ["*"] }),
  });

  console.log("clientEventsDb address", clientEventsDb.address);
  console.log(await clientEventsDb.all());

  let index = 0;

  const sayHello = async () => {
    await clientEventsDb.add(`hello from ${DIRECTORY} ${++index}`);
    setTimeout(sayHello, 2000);
  };

  clientEventsDb.events.on("join", async (peerId, heads) => {
    console.log("peer", peerId);
  });

  clientEventsDb.events.on("update", async (entry) => {
    console.log("entry", entry.payload);
  });

  await sayHello();

  process.on("SIGINT", async () => {
    await clientEventsDb.close();
    await orbitdb.stop();
    await ipfs.stop();

    process.exit();
  });
};

main();
commented

Good news and good to know about the DataStore.

I would change the following in your above script because I think the peer discovery services could be causing problems:

  const config = {
    ...libp2pDefaults(),
    addresses: {
      listen: [`/ip4/0.0.0.0/tcp/${PORT}`],
    },
    peerDiscovery: [
      mdns(),
      bootstrap({
        list: ["/ip4/127.0.0.1/tcp/8181", "/ip4/127.0.0.1/tcp/8282"],
      }),
    ],
    services: {
      pubsub: gossipsub({ allowPublishToZeroPeers: true }),
      identify: identify(),
    },
  };

To:

  const config = libp2pDefaults()
  
  config.services = { 
    identify: identify(), 
    pubsub: gossipsub({ allowPublishToZeroPeers: true })
  }
  
  config.addresses.listen = [`/ip4/0.0.0.0/tcp/${PORT}`]

  config.peerDiscovery = [ 
    mdns(), 
    bootstrap({
      list: ["/ip4/127.0.0.1/tcp/8181", "/ip4/127.0.0.1/tcp/8282"],
    })]

This means you don't have to explicitly dial the 2nd peer because mdns will help both peers discover each other (provided they are on the same local network).

commented

I don't understand what is different in both examples.in my config, mdns is already set

const config = {
    ...libp2pDefaults(),
    addresses: {
      listen: [`/ip4/0.0.0.0/tcp/${PORT}`],
    },
    peerDiscovery: [
->    mdns(),
      bootstrap({
        list: ["/ip4/127.0.0.1/tcp/8181", "/ip4/127.0.0.1/tcp/8282"],
      }),
    ],
    services: {
      pubsub: gossipsub({ allowPublishToZeroPeers: true }),
      identify: identify(),
    },
  };
commented

I just tried your config above and I'm able to get both nodes connecting to one another without explicitly dialling the first peer from the second one. I did have to delete USER1 and USER2 directories before re-running the nodes.

Using mdns() should ensure nodes can discover each other on a local network so I'm not sure why dial needs to be called explicitly. It could be something to do with the libp2pDefaults().

Anyway, if you are satisfied with the outcome then we can assume this as solved.

commented

@haydenyoung I have had the same problem when I was trying to connect two peers on two different ubuntu linux 24.04 systems across the local area network on the 192.168.0.x subnet on a home router.

I dont know how mdns works, or if this is a better issue for libp2p, and if bootstrap nodes are required, because I was to some degree hoping it wasn't necessary for peers to discover each other with dht, but i will do some more research into the problem tomorrow.