Add support for Elastic Search as a store

Question

Add support for Elastic Search as a store

jeffmcaffer opened this issue 7 years ago · comments

Elastic search seems like an interesting choice for store technology. Would give live access to the data in a queryable way.

Mark Walkom · Answer 1 · Fri Mar 09 2018 10:07:15 GMT+0800 (China Standard Time)

Not sure what this might involve, but I would be interested in discussing it and seeing what sort of resources I might be able to provide (I work @ Elastic).

Jeff McAffer · Answer 2 · Tue Mar 13 2018 11:30:08 GMT+0800 (China Standard Time)

Great @markwalkom. I should not be that bad. Basically the store API as about 5-10 methods like upsert, get, list, ... all basic point or list queries.

What are you doing with ghcrawler and perhaps I can help guide.

Mark Walkom · Answer 3 · Wed Mar 14 2018 08:53:03 GMT+0800 (China Standard Time)

I was looking at it as part of the tool set that Measure uses - https://github.com/MeasureOSS/Measure - and thought if we could enable storage from this in Elasticsearch we can also leverage Kibana for analytics :) That'd also tie nicely into other tools like https://github.com/morucci/repoxplorer and https://wiki.linuxfoundation.org/chaoss/software

…

On 13 Mar 2018 2:30 p.m., "Jeff McAffer" ***@***.***> wrote: Great @markwalkom <https://github.com/markwalkom>. I should not be that bad. Basically the store API as about 5-10 methods like upsert, get, list, ... all basic point or list queries. What are you doing with ghcrawler and perhaps I can help guide. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#98 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADCYTj7_p0qXG--fQEFx4VVqHIkIsTbXks5tdz1DgaJpZM4N2p7m> .

craigez · Answer 4 · Wed Mar 14 2018 10:36:42 GMT+0800 (China Standard Time)

Hi @markwalkom,

I have a basic implementation here: craigez@4540ca1 (and here: https://source.codeaurora.org/external/qostg/ghcrawler/commit/?h=develop&id=4540ca1283d30aa0483a9ca9adf398e0ba41772a)

It's licensed under the MIT license, but I've been unable to get permission to sign the Microsoft CLA to get the contribution merged into the upstream 😢 . Additionally, we haven't really tested it much as we will probably go an alternate direction for our metrics, but we are still looking at options.

Mark Walkom · Answer 5 · Wed Mar 14 2018 10:41:46 GMT+0800 (China Standard Time)

that's awesome, thanks @craigez! pity the licensing is causing problems though :(

craigez · Answer 6 · Wed Mar 14 2018 10:48:15 GMT+0800 (China Standard Time)

It's not the licensing, MIT is fine, just the CLA to Microsoft.

craigez · Answer 7 · Wed Mar 14 2018 10:49:26 GMT+0800 (China Standard Time)

I'll poke on this again and see if I can get it resolved soon.

k----n · Answer 8 · Sun Jan 26 2020 18:17:30 GMT+0800 (China Standard Time)

It seems like https://github.com/yougov/mongo-connector can be useful to sync to Elasticsearch