REST API Specifications

Question

REST API Specifications

arobsn opened this issue a year ago · comments

Info

Endpoint:

get /info/

Response

{
  version: number,
  mode: "full" | "light",
  height: number,                  // indexed height
  additionalIndexes: {             // only return this if mode == "light"
    contracts: string[],
    contractTemplates: string[],
    tokenIds: string[]
  } | null
}

Block Info and Statistics

Endpoints

GET /blocks/
GET /blocks/{headerId:HexString}/
GET /blocks/at/{height:int}/

URL queries

skip: int
take: int

Boxes

Endpoints

GET  /boxes/{spent|unspent|all}/{boxId:HexString}/
GET  /boxes/{spent|unspent|all}/tokens/{tokenId:HexString}/
GET  /boxes/{spent|unspent|all}/addresses/{address:string}/
GET  /boxes/{spent|unspent|all}/contracts/{contract:HexString}/
GET  /boxes/{spent|unspent|all}/contracts/templates/{template:HexString}/

URL queries

skip: int
take: int

tokenId: HexString | "empty"

R4: HexString | "empty"
R5: HexString | "empty"
R6: HexString | "empty"
R7: HexString | "empty"
R8: HexString | "empty"
R9: HexString | "empty"

confirmed: boolean

If confirmed is omitted or equal to null, then confirmed and unconfirmed boxes must be merged. Which means that new unconfirmed boxes must be added to returning array and confirmed boxes that are being spent in the mempool must be removed from the returning array.
If a default ordering is needed for take and skip, I suggest using creationHeight and boxId, as creationHeight solely is not unique.

Custom query endpoint

dApp constantly need to query for multiple contracts/addresses/templates/tokens, to potentially reduce the amount of roundtrips, I propose a custom query endpoint as described below.

Endpoint

POST /boxes/query/

Body

{
  spent: boolean | null,

  addresses: string[] | null,
  contracts: HexString[] | null,
  templates: HexString[] | null,
  contractHashes: HexString[] | null,
  templateHashes: HexString[] | null,

  tokens: HexString[] | null,

  registers: {
    R4: HexString | "empty" | null,
    R5: HexString | "empty" | null,
    R6: HexString | "empty" | null,
    R7: HexString | "empty" | null,
    R8: HexString | "empty" | null,
    R9: HexString | "empty" | null
  } | null,

  skip: number | null,
  take: number | null
}

addresses, contracts, templates, contractHashes, and templateHashes are exclusive.
At least one of addresses, contracts, templates, contractHashes, templateHashes, or tokens needs to be defined for the query to be considered valid.

Response

Array<{
  boxId: HexString,
  transactionId: HexString,
  index: number,
  ergoTree: HexString,
  creationHeight: number,
  value: string,
  assets: { tokenId: HexString, amount: string }[]
  additionalRegisters: {
    R4?: HexString,
    R5?: HexString,
    R6?: HexString,
    R7?: HexString,
    R8?: HexString,
    R9?: HexString,
  },
  state: {
    confirmed: boolean,
    spent: boolean,
  }
}>

JavaScript doesn't work well with JSON and long numbers, so box.value and box.assets.amounts must be typed as string, for improved compatibility.

Tokens

Endpoints

GET /tokens/{tokenId}/                // parsed token metadata
GET /tokens/{tokenId}/minting-box/

Node Relay Endpoints

The following endpoint will serve as proxy endpoint for the node behind running uexplorer instance.

Endpoints

GET  /node/headers/last/{count:int}/  // --> node's GET  /blocks/lastHeaders/{count}/ endpoint
GET  /node/info/                      // --> node's GET  /info/ endpoint
POST /node/transaction/               // --> node's POST /transactions/ endpoint
POST /node/transaction/check/         // --> node's POST /transactions/check/ endpoint

Ross Weir · Answer 1 · Wed Jul 05 2023 06:47:49 GMT+0800 (China Standard Time)

Looking good 👍

For the boxes endpoints - what do you think of moving all the path parameters to query params as well. Maybe something like GET /boxes?spent=true&address=...&token=...&r4=... etc - which is basically just the boxes/query endpoint.

Comparisons:

GET  /boxes/{spent|unspent}/{boxId:HexString}/ -> GET /boxes?spent=true&id={boxId:HexString}

GET  /boxes/{spent|unspent}/tokens/{tokenId:HexString}/ -> GET /boxes?spent=true&token={tokenId:HexString}

GET  /boxes/{spent|unspent}/addresses/{address:string}/ -> GET /boxes?spent=true&address={address:string}

... etc ...

By using a single endpoint we'd have less code to maintain, less tests, less in our open api spec, less overhead for API consumers since we have 1 /boxes endpoint, etc.

Currently, I can't think of a time where I'd prefer to use a specialised /boxes endpoint over just using the familiar /boxes/query for everything

At least one of addresses, contracts, templates, contractHashes, templateHashes, and tokens needs to be defined for the query to be considered valid.

Why is tokens required?

Alison Robson · Answer 2 · Wed Jul 05 2023 08:04:12 GMT+0800 (China Standard Time)

For the boxes endpoints - what do you think of moving all the path parameters to query params as well. Maybe something like GET /boxes?spent=true&address=...&token=...&r4=... etc - which is basically just the boxes/query endpoint.

Interesting point, I think that having primary query params composing the URL and secondary ones as params can make API easer to understand. However you made me realize that {spent|unspent} is a secondary param, and tokenId is both primary and secondary. So we can find a middle way and remove all */tokens/{tokenId}/ queries, add a tokenId URL param, and remove {spent|unspent} and make it a spent URL param with default value equal true. What do you think about this?

POST /boxes/query/ still required because of batched queries and/or for too long queries. We can rename it to just POST /boxes/ though.

Why is tokens required?

My mistake, it's not required. Updated the text to make clear.

Ross Weir · Answer 3 · Wed Jul 05 2023 08:18:27 GMT+0800 (China Standard Time)

Interesting point, I think that having primary query params composing the URL and secondary ones as params can make API easer to understand. However you made me realize that {spent|unspent} is a secondary param, and tokenId is both primary and secondary. So we can find a middle way and remove all */tokens/{tokenId}/ queries, add a tokenId URL param, and remove {spent|unspent} and make it a spent URL param with default value equal true. What do you think about this?

Yep this is sounding good, for this part:

and remove {spent|unspent} and make it a spent URL param with default value equal true

Would we still be able to provide spent=null or something to get both spent and unspent in one call? To keep it flexible

POST /boxes/query/ still required because of batched queries and/or for too long queries. We can rename it to just POST /boxes/ though.

IMO /boxes/query is better, as an API consumer I'd expect a POST /boxes endpoint to be a endpoint for creating box instances

For these:

GET  /boxes/{spent|unspent}/contracts/{contract:HexString}/
GET  /boxes/{spent|unspent}/contracts/{contract:HexString}/tokens/{tokenId:HexString}/

GET  /boxes/{spent|unspent}/contracts/hashes/{hash:HexString}/
GET  /boxes/{spent|unspent}/contracts/hashes/{hash:HexString}/tokens/{tokenId:HexString}/

GET  /boxes/{spent|unspent}/contracts/templates/{template:HexString}/
GET  /boxes/{spent|unspent}/contracts/templates/{template:HexString}/tokens/{tokenId:HexString}/

GET  /boxes/{spent|unspent}/contracts/templates/hashes/{hash:HexString}/
GET  /boxes/{spent|unspent}/contracts/templates/hashes/{hash:HexString}/tokens/{tokenId:HexString}/

I'm not sure I see a need for both hashed & non-hashed endpoints, when would I use one over the other?

Alison Robson · Answer 4 · Wed Jul 05 2023 08:31:36 GMT+0800 (China Standard Time)

Would we still be able to provide spent=null or something to get both spent and unspent in one call? To keep it flexible

Could't think any use case for that, but don't hurts having it as a possibility.

IMO /boxes/query is better, as an API consumer I'd expect a POST /boxes endpoint to be a endpoint for creating box instances

You are right.

I'm not sure I see a need for both hashed & non-hashed endpoints, when would I use one over the other?

@pragmaxim is going to index hashes to help on secondary indexes so we thought "why not"? But I agree with you, that has really marginal use cases, and we should probably transform them into URL query params.

Ross Weir · Answer 5 · Wed Jul 05 2023 08:36:43 GMT+0800 (China Standard Time)

@pragmaxim is going to index hashes to help on secondary indexes so we thought "why not"? But I agree with you, that has really marginal use cases, and we should probably transform them into URL query params.

Sorry if I'm being a bit pedantic on these - I've worked on APIs that have a heap of endpoints that were very similar and it makes it hard for API consumers to use our API and harder for us to maintain

What I was basically getting at was maybe we just drop non-hashed endpoints and keep the hash ones - if users have contract/template hashes they're good to go, if not they run them through a hashing func first but URL query params could also work

Alison Robson · Answer 6 · Wed Jul 05 2023 08:54:27 GMT+0800 (China Standard Time)

Updated:

Removed {tokenId} from URLs where it wasn't a primary param, and added it as a query param.
Removed */hashes/* endpoints and kept then only on /boxes/query/ endpoint.
Changed confirmation state to {spent|unspent} to {spent|unspent|all}, I think it's more clear than spent=null to get all.

Alison Robson · Answer 7 · Wed Jul 05 2023 08:58:44 GMT+0800 (China Standard Time)

Sorry if I'm being a bit pedantic on these - I've worked on APIs that have a heap of endpoints that were very similar and it makes it hard for API consumers to use our API and harder for us to maintain

Don't worry, your input is being really helpful.

What I was basically getting at was maybe we just drop non-hashed endpoints and keep the hash ones - if users have contract/template hashes they're good to go, if not they run them through a hashing func first but URL query params could also work

Hmm, probably best to keep non-hashed endpoints as it is more straight forward and it's what devs are used to do (at least for contracts), hashed endpoints requires one more step from clients.

Ross Weir · Answer 8 · Wed Jul 05 2023 14:28:23 GMT+0800 (China Standard Time)

Updated:

Removed {tokenId} from URLs where it wasn't a primary param, and added it as a query param.

Removed */hashes/* endpoints and kept then only on /boxes/query/ endpoint.

Changed confirmation state to {spent|unspent} to {spent|unspent|all}, I think it's more clear than spent=null to get all.

Having another look I think I still lean towards having a base GET /boxes endpoint with the rest being query params. If we follow API endpoint structure in terms of resources and their hierarchical relationships this is how it reads for me:

GET  /boxes/{spent|unspent|all}/tokens/{tokenId:HexString}/ -> Return a Token resource related to a specific Box resource
GET  /boxes/{spent|unspent|all}/addresses/{address:string}/ -> Return an Address resource that is related to a specific Box resource
GET  /boxes/{spent|unspent|all}/contracts/{contract:HexString}/ -> Return a Contract resource related to a Box resource 
... etc

Whereas:

GET /boxes?address={address:string}

Is clear that it's returning a Box resource by filtering a certain address

If you get my drift 🏄

Alison Robson · Answer 9 · Wed Jul 05 2023 22:29:57 GMT+0800 (China Standard Time)

Yeah, you have a solid point, I'm with you. Let's wait for @pragmaxim's feedback so we can do all changes at once.

pragmaxim · Answer 10 · Mon Jul 24 2023 14:59:39 GMT+0800 (China Standard Time)

Thanks guys, regarding the debate whether path-based or query based endpoints, imho that is just implementation detail. We should look at the API such that user is aware of the 1:M relationship between address and boxes, so it returns either a detail or aggregations of the first path element :

/boxes/{spent|unspent}/{boxId:HexString}/ - give me detail of given box
/boxes/{spent|unspent|all}/addresses/{address:string}/ - give me details of boxes for given address
/addresses/{address:string} - give me detail of given address

That would sort of work, I'm worried more about how to decide on return type, ie. give me only box ids for given address :

/boxes/{spent|unspent|all}/addresses/{address:string}/?hash=true

As sometimes it can be even tens of thousands of boxes for an address. So overall, I have a feeling that WHAT could be decided by path-based style and HOW could be decided by query-based style.

Alison Robson · Answer 11 · Wed Jul 26 2023 21:30:40 GMT+0800 (China Standard Time)

Thanks guys, regarding the debate whether path-based or query based endpoints, imho that is just implementation detail. We should look at the API such that user is aware of the 1:M relationship between address and boxes, so it returns either a detail or aggregations of the first path element

I'm happy with both approaches, I like @ross-weir's suggestion more because it will potentially lead to less code/tests to maintain. But no hard feelings.

I'm worried more about how to decide on return type

Cherry picking returning properties and checking for the existence of a resource can be very useful for some cases. My suggestion:

Support HEAD HTTP method for resource existence checking.
Example: if a user wants to fetch a specific box, then he needs to do a request like this: GET /boxes/{spent|unspent|all}/{boxId:HexString}/, but if the user only wants to check for the box existence, he needs to replace GET with HEAD so the API returns 200 if it exists, otherwise it returns 404.

Make POST /boxes/query/ accept a returning type descriptor in its body (see return below). This allows for potential query optimizations as the user will be able to fetch precisely what is needed, and the API remains concise.

{
  spent?: boolean,
  addresses?: string[],
  contracts?: HexString[],
  // ...

  return?: {
    boxId?: boolean,
    transactionId?: boolean,
    index?: boolean,
    ergoTree?: boolean,
    creationHeight?: boolean,
    value?: boolean,
    assets?: boolean,
    additionalRegisters?: boolean,
    state?: boolean
  }
}

Only properties that are equal to true will be returned. If return is omitted then all properties are returned.

pragmaxim · Answer 12 · Thu Jul 27 2023 18:27:34 GMT+0800 (China Standard Time)

Only properties that are equal to true will be returned. If return is omitted then all properties are returned.

Uff, this is exactly something that one can do in Javascript, but grow a neckbeard doing in Scala... Having any kind of dynamic dictionary or something like that in Scala is a nightmare.

You can imagine the Scala land as typesafe mapping of database entities .... That's why the whole idea with Query is really hard for me implement.

This is something that should be done by GraphQL

Ross Weir · Answer 13 · Fri Jul 28 2023 06:24:14 GMT+0800 (China Standard Time)

We should look at the API such that user is aware of the 1:M relationship between address and boxes, so it returns either a detail or aggregations of the first path element

As sometimes it can be even tens of thousands of boxes for an address. So overall, I have a feeling that WHAT could be decided by path-based style and HOW could be decided by query-based style.

You're right, the first path should be what's returned - I had them around the wrong way in my example above

For this:

That would sort of work, I'm worried more about how to decide on return type, ie. give me only box ids for given address

Agree this is probably more a GQL thing but if we want it in the API, IMO this could be it's own /boxIds endpoint which will basically just be the same as /boxes except returns ids, or add a query param to /boxes?just_ids=true since it's a common requirement

In regards to this example, just to be clear on my opinion:

/boxes/{spent|unspent}/{boxId:HexString}/ - give me detail of given box
/boxes/{spent|unspent|all}/addresses/{address:string}/ - give me details of boxes for given address
/addresses/{address:string} - give me detail of given address

I suggest a structure like:

/boxes?spent=true&..queryparams
/boxes/{boxId}?spent=true&...queryparams
/boxes?address={address}
/address/{address}

I feel like address as a query param on /boxes should reduce endpoints/code - but could be wrong as I don't have scala API experience. At the very least there will be less in our swagger spec (if we use one)

The reason I dislike having {spent|unspent|all} as a path parameter is because we overload the second path parameter if we want a simple "get a box by id" endpoint:

/boxes/{boxId}
AND
/boxes/{spent|unspent|all}/...

Thanks a lot for the discussions!

pragmaxim · Answer 14 · Fri Jul 28 2023 14:38:48 GMT+0800 (China Standard Time)

yeah, /box-ids/ is perhaps the right way...

But I'd personally keep the query-params for filtering by secondary indexes, ie. tokenId, txId, r4, r5, etc ... so the box state being part of the path is probably better ... once it becomes a mixture without rules, it's a mess