Traces not showing with Go Implementation in Query Insights

Question

Traces not showing with Go Implementation in Query Insights

kostyay opened this issue 2 years ago · comments

Hey
I've implemented this in my code.
I can see that traceparent is added to each query, unfortunately though the traces are not being linked up to my application traces. The query insights "end to end" view only shows the query breakdown without any of the application traces.
Perhaps tracespan is missing?
Any ideas how to debug this?

Subhrajyoti · Answer 1 · Sat Dec 10 2022 11:06:28 GMT+0800 (China Standard Time)

Hi @kostyay,

traceparent contains the trace-id and parent span-id, which is what is needed to link the query plans with application traces.

Now, to debug we can take a look at few things:

Can you check in your application layer what is the trace-id(part of traceparent) and is it same as the trace-id populated as part of query ?
Is the application exporting traces to GCP by using OpenTelemetry library ?

Kostya Ostrovsky · Answer 2 · Sat Dec 10 2022 22:33:14 GMT+0800 (China Standard Time)

Hey
Thanks for the reply.
Yes the traceids match and the application is reporting the traces using opentelemetry and they show in the Trace view of the console.
However Query Insights traces don't link up with the application traces.
The trace id and span that shows for the query is not one I recognize (I added verbose logging to our app printing the trace ids and the ones in QI were never printed)

kapil verma · Answer 3 · Sun Dec 11 2022 15:51:30 GMT+0800 (China Standard Time)

Hey ... which database are you using MySQL or PG?

Kostya Ostrovsky · Answer 4 · Sun Dec 11 2022 17:41:10 GMT+0800 (China Standard Time)

Postgres 14.
Do I need to enable any extensions for this to work (besides Query Insights)?

kapil verma · Answer 5 · Sun Dec 11 2022 17:50:04 GMT+0800 (China Standard Time)

Postgres 14.

Thanks. Will look into this.

Do I need to enable any extensions for this to work (besides Query Insights)?

No, Query Insights extension should be enough

Kostya Ostrovsky · Answer 6 · Sun Dec 11 2022 17:58:45 GMT+0800 (China Standard Time)

Just as a reference it doesn't work with Postgres 11 too.
We have a support ticket with Google about this, if you want some links/trace ids I can email them to you

Subhrajyoti · Answer 7 · Wed Dec 14 2022 17:02:28 GMT+0800 (China Standard Time)

@kostyay

I can see that traceparent is added to each query

Can you please give more information on how you checked that traceparent is populated for each query i.e.

By using application logs or database logs ? OR
traces from Query Insights UI ?

Kostya Ostrovsky · Answer 8 · Wed Dec 14 2022 17:46:39 GMT+0800 (China Standard Time)

I enabled debug logging in pgx driver that forces it to print all the queries before its running and saw /traceparent=xxx/ on the queries
I also saw them in the postgres.log file in the CloudSQL instance.