rapidsai / node

GPU-accelerated data science and visualization in node

Home Page:https://rapidsai.github.io/node/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Write endpoint that returns only n points.

thomcom opened this issue · comments

To time the speed of the interface and to build the first step of building a point-budget based demo, I need an endpoint that only returns n points.

My first impulse is to create a fastify plugin that modifies the API and allows for endpoints and bounds to be specified. I'm not sure that a fastify plugin can change the endpoint schema, however.

Since only particles/:table will depend on this, it is surely faster just to modify its schema and handler to this purpose. I'll write this endpoint in a test-driven fashion, starting with the test then adding to the endpoint.

  • Viz with 'max zoom out view' (e.g. all data 'visible')

  • We have a set 'maximum # of points able to render' (this is usually lower than full dataset)

  • Currently writing this request, for the server to return a fixed number of points.

  • How do we decide and keep track of what points to show from the full dataset?

  • I originally planned on having the client do all of the point specifications, but it would be interesting to use the server for tracking what points have been displayed. It would be trivial to add metadata columns to the DataFrame about which points have been displayed, their time of display, and which request gathered them.

Thinking about this endpoint, how does it sample which points are to be returned?

If no bounding box is passed, then it will randomly sample n points from the full dataset.

If a bounding box is also passed, then it will filter within the bounding region, then if the number of points is still greater than npoints, it will sample those.

I'm noticing that there's no random functionality in our gpu code at this time, whenever a sample is needed in python space then cupy is used. Looking into a solution....