Report search index size metrics

[Ovais Tariq]

Similar to collection size metrics, we need to report the search index size metrics

[Akhil]

@ovaistariq Can I work on this?
I went through the metrics code. Got a sense of how it has been done for collection size metrics.

• We can collect system_disk_used_bytes from the mertics API of typesense. But this is for the whole search cluster and Typesense doesn't provide any information on usage by index ( collection in Typesense). Is there a way we can get the size by index?
• Typesense Collection Retrieve provides the details on a number of documents in the index. Is it useful to report?

What are all tags we are looking to report ?
tigris_tenant, tigris_tenant_name, project, db, branch, index

[Himank Chaudhary]

@akhill10 We are managing this information on our side in a metadata table. @efirs Could you please add here what is missing and what is needed to complete this?

[Yevgeniy Firsov]

Hi, @akhill10!
Thanks for your interest in contributing!

As a first step we need to implement search counter part tenant.CollectionSize.
Where we need to use tenant.Encoder.EncodeFDBSearchTableName(index.StoreIndexName()) as a KV store table name.
Also we need to add tenant.ProjectSearchSize() and tenant.SearchSize(),
this can be done using tenant.ListSearchIndexes similar to tenant.DatabaseSize() and tenant.Size().

Then we need to wire it in the quote package where we emit metrics.
See storage.updateMetricsForNamespace.

Let me know if you need more information.

[Akhil]

Thanks @himank @efirs , that was very helpful. Raised PR please take a look.

[Ovais Tariq]

Fixed by #1113