Giters

fairDataSociety / fdp-storage

Serverless Web3 filesystem for organizing users' personal data implemented in Typescript.

Home Page:https://www.npmjs.com/package/@fairdatasociety/fdp-storage

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Newly created pod can't be fetched

tomicvladan opened this issue · comments

commented

This issue occurres when using Blossom in Fairdrive. After creating a pod, the pod list on the left side is automatically refreshed, but the new pod doesn't appear in the list.

To confirm that the error doesn't come from Fairdrive or Blossom, this line of code is relevant:
https://github.com/fairDataSociety/blossom/blob/a4f3e3087dee02b45916b0b70c48347cd7c3e366/src/services/fdp-storage/fdp-storage-access.ts#L138

Here the service worker script invokes appropriate methods from fdp-storage. If responses are logged out, there will be an entry for creating a pod and after it finishes another entry will appear which returns list of pods. But the list pod won't include the new pod there.

commented

Just to add recent findings. I reproduced this issue in Fairdrive directly, without Blossom.

Steps to reproduce:

  1. Compile browser version of fdp-storage from fix/bee-js-new-762 branch.
  2. Replace the compiled index.browser.min.js in node_modules of Fairdrive
  3. Remove the .next folder to remove cache
  4. In src/context/FdpStorageContext.tsx file, disable caching by setting isUseCache: false
  5. Run Fairdrive, log in and create a pod

Pod list will refresh but without new pod.

commented

One more note, this issue happens only on bee-1. When using fdp-play it works fine.

commented

@tomicvladan @darkobas2

The error is also sometimes reproduced in Fairdrive with "@fairdatasociety/fdp-storage": "^0.11.0" and the cache is disabled (isUseCache: false).
After a manual test, it turned out that the problem is due to the fact that the bee-1 server responds with old data for the chunk (404 error instead of current data). After some time (seconds) the data becomes available.

Apparently, the problem may be in the caching of the chunk request OR in the correctness of the responses of the Bee node.

The chunk containing new data is requested before the data is written there, so this may be the reason for caching the Bee node's responses. Although the node can make chunk data available after some time if the Bee node has been updated with some new mechanisms.

It is required to test the server side:

If this behavior is on the side of the Bee node, then it is worth planning changes in the fdp protocol that could detect the moment when the data becomes available.

commented

@darkobas2 this is probably connected with serverside caching, chunk is in cache and is not being updated. Can you disable caching on gateway so we can test this and eliminate potential issue ?

commented

We tested this with caching disabled. Still exists.

commented

I am attaching a har with all the site requests. har can be opened on this site https://toolbox.googleapps.com/apps/har_analyzer/.

Check the POST request to https://bee-1.fairdatasociety.org/soc/c61a782953dec13ec9614469a2801b029378ee48/cf29db53d5c44182b4c8b222b1e0036eb48bbedd3b6087ae80426e9e275774aa?sig=58d35bd3d67619ca18756c3e1487a521c7d7a8275baf5a3becd442b1aaeae8d3524cf784501dc43f27c77391c76e062d3db8f1ca7b3dc93daa2de49098c708c51b


It returns {"reference":"d70c30039aafcc857d18ddaf35c6339ca04a5357b9b7bf992a5be8b5b143a055"}

Then there is a GET to https://bee-1.fairdatasociety.org/chunks/d70c30039aafcc857d18ddaf35c6339ca04a5357b9b7bf992a5be8b5b143a055 which returns {"code":500,"message":"read chunk failed"}

After a few seconds the data becomes available

soc-error.har.zip