Clickthrough URLs with $__field_ variables don't work with Elasticsearch data source

Question

Clickthrough URLs with $__field_ variables don't work with Elasticsearch data source

amotl opened this issue 4 years ago · comments

I am using 0.9.0 map panel with Grafana 6.5.2.

I am trying to pass the URL as below

https://xyz.com/d/xyz-monitor/xyz-monitor?&from=${__from}&to=${__to}&var-Filter=clientGeo|=|${__field_clientGeo}&var-Filter=applicationDescription|=|${__field_applicationDescription}

In the above URL, clientGeo is the geo hashpoint and applicationDescription is another field. Both fields coming from elastic search query.

I can see __from and __to built in variables picking up the time fields form the main panel but clientGeo and applicationDescription are not resolving to values. Both clientGeo and applicationDescription fields are in the elastic query and returning data to main panel as I can see data on map panel, but not getting resolved to values in the click through URL.

Looking at he clickthrough URL information I saw that we need to prefix with $__field_ with each data field in URL, but it does not seem to be working or I am doing something wrong. Any help is appreciated.

thanks

Andreas Motl · Answer 1 · Fri Mar 13 2020 23:45:09 GMT+0800 (China Standard Time)

@vmanju123 added a remark:

The same issue is being discussed within #12.

Andreas Motl · Answer 2 · Sat Mar 14 2020 00:16:39 GMT+0800 (China Standard Time)

Dear @vmanju123,

thanks for writing in. Apparently, this feature is only implemented within setTableValues, so it will probably only work when using data in table format. Bummer.

https://github.com/panodata/grafana-map-panel/blob/315f889f49d0d9ecc7381cd4804dbc3a8261d6ea/src/data_formatter.ts#L352-L357

Am I correct to assume data from Elasticsearch is not using table format?

With kind regards,
Andreas.

P.S.: If only #56 would work for you. Then you could add this feature on your own behalf and send a pull request so that we could comfort @rsaifulin, @weedaj and @krzysion, who are probably observing the same issue (#12).

vmanju123 · Answer 3 · Sat Mar 14 2020 03:35:44 GMT+0800 (China Standard Time)

I can confirm now that it is indeed working for Table format, since i have time series data in es database

Andreas Motl · Answer 4 · Mon Mar 16 2020 02:35:22 GMT+0800 (China Standard Time)

78882ec adds a test for checking "clickthroughUrl" interpolation when using table format.

Andreas Motl · Answer 5 · Mon Mar 16 2020 02:54:10 GMT+0800 (China Standard Time)

I can confirm now that it is indeed working for Table format, since i have time series data in es database.

Does this mean you have been able to make it work by switching to "table format" while still using the same Elasticsearch data source as before?

Andreas Motl · Answer 6 · Tue Mar 17 2020 10:00:13 GMT+0800 (China Standard Time)

Dear @vmanju123,

I haven't used the Elasticsearch location data source - as of "geohash: An ElasticSearch query that returns geohashes" - yet. Now, c800ca3 improves the test suite by adding tests for checking the corresponding code path through setGeohashValues().

On top of that, 99f1810 adds the __field_ interpolation feature, now also for the Elasticsearch data source.

Would that [also] improve your situation? As we are not using Grafana with Elasticsearch, I can't tell if this makes any sense and will be happy about any feedback from you. Please let me know when I should build a new release which includes these changes.

With kind regards,
Andreas.

cc @rsaifulin, @weedaj, @krzysion

vmanju123 · Answer 7 · Thu Jun 25 2020 00:07:12 GMT+0800 (China Standard Time)

as i mentioned i am using the _field variables in URL and its working fine for my elastic queries

Rob Oudendijk · Answer 8 · Fri Jul 24 2020 18:06:16 GMT+0800 (China Standard Time)

vamnju123

Do you have online dashboard with an example? We at Safecast tried to use the $__field variables but that did not work. See attached screenshots.

weedaj · Answer 9 · Mon Aug 03 2020 16:01:25 GMT+0800 (China Standard Time)

hi, I'm still struggling. Could you give me an explicit example in below situation? I'm running an Elastic query: [image: image.png] The map populates fine. I need the bcc.sc.location.keyword (set as 'Label field') value set as var-location in the clickthrough URL: /v7/d/MYDASHBOARD/&var-location=XXX With that given, what would I need to populate in respectively 'Link field' and 'Clickthrough URL'? Thanks in advance for your help. Jeroen Op wo 24 jun. 2020 om 18:07 schreef vmanju123 <notifications@github.com>:

…

as i mentioned i am using the _*field* variables in URL and its working fine for my elastic queries — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#57 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAM2BRBGRR3IODIAZ7TDSLLRYIQD7ANCNFSM4LHFCU3Q> .

Rob Oudendijk · Answer 10 · Tue Sep 15 2020 23:12:59 GMT+0800 (China Standard Time)

Any updates on the issue?

Rob Oudendijk · Answer 11 · Wed Oct 21 2020 17:52:41 GMT+0800 (China Standard Time)

Any updates on the issue?

Andreas Motl · Answer 12 · Wed Oct 21 2020 19:00:49 GMT+0800 (China Standard Time)

Dear Rob,

I believe it would work with data coming from InfluxDB in table format as outlined within #57 (comment) already. We are using InfluxDB, so this was obvious to have at the first place.

Apart from that, I've added support for data coming from Elasticsearch the other day, see #57 (comment).

Apparently, @vmanju123 confirmed it worked for them. However, I only see now that @matschaffer replied at 99f1810#commitcomment-40707078 and that he might also be struggling. Sorry to hear that.

Maybe you can share more details from the edit screen of your dashboard?

With kind regards,
Andreas.

Rob Oudendijk · Answer 13 · Thu Oct 22 2020 10:34:49 GMT+0800 (China Standard Time)

Andreas, Thank you for your reply. What we try to do is open a new tab with detail information about sensors. The links for the click-through work fine in tables, but not in the map click-throughs. See attached screenshot how we implemented the setup. If you need more detail information, do not hesitate to ask us. We hope this would help you to troubleshoot the issue. Regards, Rob Oudendijk Regards, Rob Oudendijk Yuka Hayashi http://yr-design.biz http://oudendijk.biz http://about.me/robouden tel +81 80-22605966 Skype: robouden Facebook:robouden <http://on.fb.me/QeKw2P> linkedin:robouden <https://www.linkedin.com/in/roboudendijk>

…

On Wed, Oct 21, 2020 at 8:01 PM Andreas Motl ***@***.***> wrote: Dear Rob, I believe it would work with data coming from InfluxDB in table format as outlined within #57 (comment) <#57 (comment)> already. We are using InfluxDB, so this was obvious to have at the first place. Apart from that, I've added support for data coming from Elasticsearch the other day, see #57 (comment) <#57 (comment)> . Apparently, @vmanju123 <https://github.com/vmanju123> confirmed it worked for them. However, I only see now that @matschaffer <https://github.com/matschaffer> replied at 99f1810#commitcomment-40707078 <99f1810#commitcomment-40707078> and that he might also be struggling. Sorry to hear that. Maybe you can share more details from the edit screen of your dashboard? With kind regards, Andreas. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#57 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKSVNSKE7OEQNYW72PYEUTSL25PDANCNFSM4LHFCU3Q> .

Andreas Motl · Answer 14 · Thu Oct 22 2020 15:42:07 GMT+0800 (China Standard Time)

Hi Rob,

See attached screenshot how we implemented the setup.

Can not see it.

Cheers,
Andreas.

Rob Oudendijk · Answer 15 · Thu Oct 22 2020 16:11:10 GMT+0800 (China Standard Time)

Rob Oudendijk commented 4 years ago

Rob Oudendijk · Answer 16 · Thu Oct 22 2020 16:13:20 GMT+0800 (China Standard Time)

Andreas,

Sorry. I replied in an email with attachments. Seems that did not wok for GitHub.
Uploaded them here just before.

Regards,
rob

Andreas Motl · Answer 17 · Thu Oct 22 2020 16:25:42 GMT+0800 (China Standard Time)

Thanks Rob. Is $device_urn coming from a Grafana Variable or from the database? If it is coming from the database, you probably should use $__field_device_urn there within the Clickthrough URL.

Rob Oudendijk · Answer 18 · Thu Oct 22 2020 16:30:09 GMT+0800 (China Standard Time)

Andreas,

We tried both with and without. Same results.
Any other idea how to fix it?

regards,
rob

Rob Oudendijk · Answer 19 · Thu Oct 22 2020 16:35:25 GMT+0800 (China Standard Time)

Andreas,

If we click on the hovering point the page opens with the following URL: instead of supplying the data from the requested field. On the table is works fine.

regards,
rob

Andreas Motl · Answer 20 · Thu Oct 22 2020 16:53:28 GMT+0800 (China Standard Time)

Sorry for that. Maybe this is coming from the specific style of data source you are using here. There are many intertwingulating things within the code base...

Andreas Motl · Answer 21 · Thu Oct 22 2020 16:59:59 GMT+0800 (China Standard Time)

Hi Rob,

I just reviewed the changelog [1] and found

Interpolate datapoint field values with __field prefix into transformation data, now also for elasticsearch queries. (99f1810)

apparently not to be part of an official release yet. Are you running a custom build or did you just install 0.9.0? If so, that might be an easy explanation why that would not work for you, yet.

With kind regards,
Andreas.

[1] https://github.com/panodata/grafana-map-panel/blob/develop/CHANGELOG.md

Rob Oudendijk · Answer 22 · Thu Oct 22 2020 17:52:33 GMT+0800 (China Standard Time)

Andreas,

We are running 0.9.0 as far as I know. I will check to be sure.
Sorry for my innocent, but how to run a custom build?

regards
Rob Oudendijk

Rob Oudendijk · Answer 23 · Thu Oct 22 2020 18:00:09 GMT+0800 (China Standard Time)

OOPs. maybe not true.

We are running a script :/usr/sbin/grafana-cli --pluginUrl https://github.com/panodata/grafana-map-panel/archive/4f4765b68b2c02ca7de2b36dcc24349b5e10ef47.zip plugins install grafana-worldmap-panel-ng

not sure what version that is.

regards,
rob

Rob Oudendijk · Answer 24 · Thu Oct 22 2020 18:43:10 GMT+0800 (China Standard Time)

Andreas,

Installed the latest development code from https://github.com/panodata/grafana-map-panel/archive/develop.zip and restarted the Grafana server. Results are still the same.

Any other suggestions?

BTW how to check what version of the plugin the browser is using?

regards,
rob

Andreas Motl · Answer 25 · Thu Oct 22 2020 19:46:08 GMT+0800 (China Standard Time)

Dear Rob,

downloading the sources will definitively not work. You will have to build/compile them. If you dare to, you might find some pointers how to do that within [1].

Sorry that I am too busy right now to deliver that for you. Please let me know how that goes for you in general. On the other hand, I will try to come back to giving some love to that repository again. There are also some great contributions from @JonasProgrammer still lingering within pull requests.

With kind regards,
Andreas.

[1] https://github.com/panodata/grafana-map-panel/blob/develop/CONTRIBUTING.md

Andreas Motl · Answer 26 · Thu Oct 22 2020 19:55:25 GMT+0800 (China Standard Time)

BTW how to check what version of the plugin the browser is using?

I don't know from the top of my head. Maybe that command will also display the plugin version(s)?

grafana-cli plugins ls

Rob Oudendijk · Answer 27 · Thu Oct 22 2020 19:59:40 GMT+0800 (China Standard Time)

Andreas, I did run the command and the reply: *installed plugins:* *dalvany-image-panel @ 2.1.1* *grafana-googlesheets-datasource @ 1.0.0* *grafana-image-renderer @ 2.0.0* *grafana-worldmap-panel @ 0.3.2* *grafana-map-panel @ 0.9.0* *petrslavotinek-carpetplot-panel @ 0.1.1* *simpod-json-datasource @ 0.1.7* regards, rob Regards, Rob Oudendijk Yuka Hayashi http://yr-design.biz http://oudendijk.biz http://about.me/robouden tel +81 80-22605966 Skype: robouden Facebook:robouden <http://on.fb.me/QeKw2P> linkedin:robouden <https://www.linkedin.com/in/roboudendijk>

…

On Thu, Oct 22, 2020 at 8:55 PM Andreas Motl ***@***.***> wrote: BTW how to check what version of the plugin the browser is using? I don't know from the top of my head. Maybe that command will also display the plugin version(s)? grafana-cli plugins ls — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#57 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKSVNWDAOCED2UAN7XJEPDSMAMT5ANCNFSM4LHFCU3Q> .

Andreas Motl · Answer 28 · Thu Oct 22 2020 20:05:32 GMT+0800 (China Standard Time)

I did run the command

Excellent. Now we know how one would find out about that.

However, to run the most recent development build, you will have to follow the guidelines referenced at #57 (comment).

Also, you might want to uninstall the original grafana-worldmap-panel. While I can't remember this detail exactly, it could be that they get in the way of each other (shouldn't happen, but still worth a try).

Rob Oudendijk · Answer 29 · Thu Oct 22 2020 20:23:23 GMT+0800 (China Standard Time)

Andreas, Trying to find the guidelines referenced at #57 (comment) <#57 (comment)>., but I can not find them. You have a link? regards rob Regards, Rob Oudendijk Yuka Hayashi http://yr-design.biz http://oudendijk.biz http://about.me/robouden tel +81 80-22605966 Skype: robouden Facebook:robouden <http://on.fb.me/QeKw2P> linkedin:robouden <https://www.linkedin.com/in/roboudendijk>

…

On Thu, Oct 22, 2020 at 9:05 PM Andreas Motl ***@***.***> wrote: I did run the command Excellent. Now we know how one would find out about that. However, to run the most recent development build, you will have to follow the guidelines referenced at #57 (comment) <#57 (comment)> . Also, you might want to uninstall the original grafana-worldmap-panel. While I can't remember this detail exactly, it could be that they get in the way of each other (shouldn't happen, but still worth a try). — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#57 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKSVNQMRKVQC2K7JDB2DNTSMANZXANCNFSM4LHFCU3Q> .

Andreas Motl · Answer 30 · Thu Oct 22 2020 20:27:45 GMT+0800 (China Standard Time)

You have a link?

Downloading the sources will definitively not work. You will have to build/compile them. If you dare to, you might find some pointers how to do that within [1].

[1] https://github.com/panodata/grafana-map-panel/blob/develop/CONTRIBUTING.md

Mat Schaffer · Answer 31 · Fri Oct 23 2020 22:33:59 GMT+0800 (China Standard Time)

hi @amotl - thanks for the help so far. I downloaded and built 3f77d55 and it's the same behavior (looks like maybe some CSS is a little off and src/data_formatter.test.ts throws prettier warnings). Here's what it looks like.

The clickthrough url is set to /d/DFSxrOLWk/safecast-device-details?orgId=1&from=now-2d&to=now&var-device_urn=$__field_device_urn

and if I click on a point I get http://grafana.127.0.0.1.nip.io:8000/d/DFSxrOLWk/safecast-device-details?orgId=1&from=now-2d&to=now&var-device_urn=$__field_device_urn&refresh=15m

Looks like the replacement just doesn't happen at all.

Mat Schaffer · Answer 32 · Fri Oct 23 2020 23:01:01 GMT+0800 (China Standard Time)

ah.... looks like I didn't have the docker set up quite right. I think that's probably 0.9.0 still

Mat Schaffer · Answer 33 · Sat Oct 24 2020 16:15:04 GMT+0800 (China Standard Time)

Got the build sorted and sure enough, click through works!

Any chance of getting a pre-release? I could set up a fork with a github action, but maybe good for other people to be able to use it too.

Mat Schaffer · Answer 34 · Sat Oct 24 2020 17:33:05 GMT+0800 (China Standard Time)

Actually who needs a proper pipeline? ;) I pushed a zip to https://github.com/matschaffer/worldmap-panel/releases/tag/0.10.0-pre1 and popped it up on http://grafana-dev.safecast.cc/d/u5-KaDEWk/air-data?orgId=1

Rob Oudendijk · Answer 35 · Sat Oct 24 2020 18:22:41 GMT+0800 (China Standard Time)

Nice job!! . One more request. to get multiple values on the hover panel. That would be a big help for me and probably also other people.

Andreas Motl · Answer 36 · Tue Oct 27 2020 02:39:45 GMT+0800 (China Standard Time)

Dear @robouden and @matschaffer,

thanks a bunch for all your involvement to check the current state of the repository and for making an interim release. Sorry for missing out to bring the required features to an official release.

Now, the official release 0.10.0 has been made, see https://github.com/panodata/grafana-map-panel/releases/tag/0.10.0.

With kind regards,
Andreas.

P.S.:

One more request. to get multiple values on the hover panel. That would be a big help for me and probably also other people.

Let's track this within another issue? Maybe #28 already reflects what you are asking for?

Rob Oudendijk · Answer 37 · Tue Oct 27 2020 08:49:42 GMT+0800 (China Standard Time)

Andreas,

Yes, that is the same request as #28.

regards,
rob

Andreas Motl · Answer 38 · Tue Oct 27 2020 09:02:47 GMT+0800 (China Standard Time)

Thanks! Closing this now.