trace not rendered in the UI with over 30000 TRACE_COUNTER data points

[Ting Zhou]

Hi,

I'm trying to use the SDK. When I use TRACE_COUNTER, with 20K data points, the trace can be rendered in the UI. While with 30K data points, it won't render anything.

Is there any hidden constraint/threshold in the code for TRACE_COUNTER and TRACE_EVENT?

Thanks,

[Primiano Tucci]

Can you please attach a trace?
It's hard to act on this bug without more data.

There is no hardcoded limit I can think of. This must be some more subtle bug.

First step would be checking if

1. The data is there in the trace (via tools/traceconv text) and it's a trace_processor import bug
2. The data is there in the trace_processor (trace_processor_shell tracefile; query the counter table) and it's a UI bug

Check the above and report back.

[Ting Zhou]

Hi, @primiano

I created a sample project here https://github.com/ztlevi/perfetto-trace-example/.
By changing this line's 30000 to 20000 https://github.com/ztlevi/perfetto-trace-example/blob/main/data_generator_test.cc#L68, the generated example.pftrace can be opened with UI. While 30000 does not.

I'm not sure what the column names are. select c.ts, c.value, t.name from counter doesn't work...

You can find the example.pftrace in the repo

[Ting Zhou]

BTW, I not sure what's the proper way to update the end time of the trace. So currently it's ugly coded as https://github.com/ztlevi/perfetto-trace-example/blob/main/data_generator_test.cc#L46-L50

[Ting Zhou]

Hi, The timestamps are not the same. They have small differences.

…

On Jan 20, 2022, 02:31 -0800, Primiano Tucci ***@***.***>, wrote: The trace is full of counters that have the same timestamp packet { trusted_uid: 0 timestamp: 475458015201681 trusted_packet_sequence_id: 2 track_event { type: TYPE_COUNTER track_uuid: 11364617305446416303 double_counter_value: 1.0150790214538574 } sequence_flags: 2 timestamp_clock_id: 3 } packet { trusted_uid: 0 timestamp: 475458015301681 trusted_packet_sequence_id: 2 track_event { type: TYPE_COUNTER track_uuid: 11364617305446416303 double_counter_value: 1.0159167051315308 } sequence_flags: 2 timestamp_clock_id: 3 } It's 475458121101681 all over the places. There's something wrong with your sources. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[Lalit Maganti]

The problem here is reasonably straightforward: track event tries to be efficient by emitting as little as possible on each event. To do this, it emits some "interning" state at the start of the trace which contains information like the track name and type.

Your buffer size is 1024KB. This is big enough to hold 20K events without wrapping but for 30K events, you wrap the ring buffer so you lose the "interned" state at the start of the trace. Without this interned state, trace processor will simply skip your events as it does not know how to parse them.

To fix this you have a few options:

1. Increase your buffer size to something which can hold 30K events (or however many events you want to store).
2. Change you buffer mode to DISCARD - this means the trace will essentially stop when you hit the buffer size
3. Turn on "write_into_file": this flushes your trace to disk every few 100ms so that you won't lose data when wrapping
4. Set IncrementalStateConfig.clear_period_ms so that you periodically re-emit the "interned" state - this will still mean. that you will lose some events until trace processor sees this packet in the trace.

[Ting Zhou]

@LalitMaganti Thanks. It works like a charm.

May I ask is there any way to grab the actual perfetto::protos::Trace object instead of the raw chars?

I'm using arbitrary time so the trace's end time needs to be modified in order to visualize in the UI. Currently I have to ParseFromString and then set_timestamp. I wonder if it's possible to handle that with perfetto SDK?

[Lalit Maganti]

It's not clear to me why you're trying to manually set the end timestamp to be something other than what's in the trace? Why not just emit the end timestamp in the trace itself?

[Ting Zhou]

I'm trying to use the dumped data from other performance test apps and it has arbitrary time. So we cannot use perfetto SDK in a hook based approach. In that case, we need to set the timestamp of the event manually

[Lalit Maganti]

If you really have to do this, the best way is to create a new TracePacket with your fake end timestamp timestamp, place that packet inside it a perfetto::protos::Trace, serialize it and concatenate the bytes to the end of the trace produced by Perfetto.

As everything is a protobuf, simply concatenating the bytes is good enough without changing the contents of the Perfetto trace stream itself.

[Ting Zhou]

@LalitMaganti Thanks. I'll try to concatenate the bytes.

I was trying to create some trace event with something like this

    TRACE_EVENT_BEGIN(category, name, track, start_time);
    TRACE_EVENT_END(category, track, end_time);

But end up having all the slices in the same color. How can we control the colors?

The slices are under different categories.

[Ting Zhou]

I found the color is related with the category's first character... Is there a way to manually control it?

[Lalit Maganti]

The color is chosen based on a hash of the slice name: https://cs.android.com/android/platform/superproject/+/master:external/perfetto/ui/src/common/colorizer.ts;l=153?q=f:perfetto%20color

There is no way to customize this at the moment nor do I know of any plans to introduce this feature in the near future.