Decentralised network: number_gte

[Adam Fuller]

When polling subgraphs on the network via the gateway, clients might receive responses from indexers at different blocks for different queries. In particular it is not desirable for a client to travel backwards in time (when polling the subgraph for updates, for example).

Developers can ensure that they don't go backwards by passing a number_gte argument to the block parameter, as follows:

async function updateProtocolPaused() {
  // It's ok to start with minBlock at 0. The query will be served
  // using the latest block available. Setting minBlock to 0 is the
  // same as leaving out that argument.
  let minBlock = 0

  for (;;) {
    // Schedule a promise that will be ready once
    // the next Ethereum block will likely be available.
    const nextBlock = new Promise((f) => {
      setTimeout(f, 14000)
    })

    const query = `
        {
            protocol(block: { number_gte: ${minBlock} }  id: "0") {
              paused
            }
            _meta {
                block {
                    number
                }
            }
        }`

    const response = await graphql(query)
    minBlock = response._meta.block.number

    // TODO: Do something with the response data here instead of logging it.
    console.log(response.protocol.paused)

    // Sleep to wait for the next block
    await nextBlock
  }
}

Docs

Graph Client should offer this as an easy option for developers, so they don't have to implement the logic themselves.

[Dotan Simha]

Few implementation details questions:

• Since we are going to offer the option to use multiple indexers (for example, with fetch strategies: fallback, race, or with client-side composition where you have a unified Query type) - how should it work in that case? I guess we'll need to maintain a record for the block number per each source?

• Do you think this should be built-in in the client? (when you run a query, we can automatically add the _meta fields to every outgoing query, and then automatically use the value returned for the next calls)

[Dotan Simha]

Also, @ardatan ^ what do you think?

[Adam Fuller]

Since we are going to offer the option to use multiple indexers

This is already how it works via the Gateway (the gateway maintains the record that you describe). In the future graph-client can facilitate going directly to multiple indexers (not via the gateway), but at the outset I think we just need to handle the number_gte part

Do you think this should be built-in in the client?

I think it does make sense to build this in as an option, I am not sure at what level of configuration makes sense

[Dotan Simha]

I think it does make sense to build this in as an option, I am not sure at what level of configuration makes sense

Yeah, I was thinking maybe this could be done on the graphql handler level, where you do something like:

sources:
  - name: uniswapv2
    handler:
      graphql:
        endpoint: https://api.thegraph.com/subgraphs/name/uniswap/uniswap-v2
        blockTracking: true # this one 

And then it will make sure to keep that source in sync (by adding _meta to queries, and use 0 by default until we know the actual value). This way users can choose what indexers/sources/Subgraphs need that.

In the future graph-client can facilitate going directly to multiple indexers (not via the gateway), but at the outset I think we just need to handle the number_gte part

If I understand correctly, users who run graph-node only, don't necessarily have the GW, right? so I guess the ideal setup would be to allow users to opt-in to that mechanism, based on their needs/sources configurations.

[Adam Fuller]

If I understand correctly, users who run graph-node only, don't necessarily have the GW, right? so I guess the ideal setup would be to allow users to opt-in to that mechanism, based on their needs/sources configurations.

Yes, this definitely should be at the user's discretion. I think the graphQL handler level you describe seems like a good option!

[Dotan Simha]

Implemented in: #37
We hope to get this in soon :)

[Dotan Simha]

This is now available in @graphprotocol/client-cli@0.0.5, docs are here: https://github.com/graphprotocol/graph-client/blob/main/packages/block-tracking/README.md